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FOREWORD 


The author of this Bulletin, Dr. R. W. B. Jackson, after graduating 
with first-class honours in Mathematics from the University of Alberta, 
continued his studies in the University of London, England. There 
he worked for five years under R. A. Fisher, Egon Pearson and others, 
and became fully seized of the value of the new statistical techniques 
they had developed. Consequently, when he joined our staff in the 
spring of 1989, I suggested that he should prepare a bulletin which 
would make available to research students in education the statistical 
techniques associated with Fisher's name. What I had in mind was 
a publication that would do for educators what Snedecor's Calculation 
and Interpretation of Analysis of Variance and Covariance had done 
for agriculturalists. This latter work, using data from biological 
experiments, had expounded Fisher's techniques of analysis of variance 
and covariance in the solution of agricultural problems. Could not 
the data from educational tests and research, which we had in abund- 
ance in our Department, be put to similar use? Research students 
in education should be shown by practical demonstration when and 

*how to use the variance and covariance method of Fisher. 

As soon as the Canadian Council for Educational Research was 
established, I applied, on Dr. Jackson's behalf, for a modest grant 
to aid this particular project. The application was well received and 
a grant-in-aid was made. For the grant, grateful acknowledgement 
is hereby made. 2 

In my first enthusiasm for the project, I envisaged a bulletin 
that would be understandable by a person with a modicum of mathe- 
matical and statistical knowledge, but I am now obliged to confess 
that Fisher's contributions to statistics cannot be explained in mathe- 
matical words of one syllable. Nevertheless, Dr. Jackson has succeeded 
in this bulletin in showing, how the illustrative problems are worked 
out step by step, and what deductions can rightfully be made from 
the results. Practically all the types of cases are given in which the 
technique of analysis of variance can be used on educational data. 
In the last chapter, dealing with analysis of covariance, only data 
exhibiting linearity of regression are considered, but similar methods 
may be used in other cases. 

I feel that this short, practical work of Dr. Jackson's should 
find a warm welcome among research workers in education. In any 
case, if it extends the knowledge of a valuable statistical technique 
among them, the bulletin will have served its purpose. 


PETER SANDIFORD, Director. 


University of Toronto 
January 1940. 
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CHAPTER I 


INTRODUCTION AND OUTLINE OF THE THEORY UNDER- 
LYING THE STATISTICAL TESTS USED IN THE 
ANALYSIS OF VARIANCE AND COVARIANCE 


The statistical method known as the Analysis of Variance and 
Covariance is one of the most useful and powerful that has been 
developed in recent years. It was first suggested and used by Professor 
R. A. Fisher in England (in 1923) and is at present one ог the standard 
methods of procedure for analyzing the results of agricultural and 
biological experiments. Its usefulness, however, is not limited to 
these fields, as it is a method which seems to admit of general appli- 
cation, and it should prove to be of particular value in the field of 
education.! It is the purpose of the present Bulletin to outline the 
general principles underlying the method and to show, by examples, 
how it may be applied to particular problems in the educational 
field. 

The method consists, as the name implies, in the breaking-up of 
the total variance (or covariance) into independent parts which may 


1The method has been used in a few cases but it is certainly not as yet in general 
use. Two interesting articles have been published since this bulletin was written: 
the first by Paul L. Dressel, on “Тһе Effect of High School on College Grades", 
and the second by Jack W. Dunlap, ‘‘Applications of Analysis of Variance to 
Educational Problems." Dunlap’s article gives a brief discussion of the Analysis 
of Variance method and a summary of the results given in articles in which this 
method has been applied to educati€nal problems. A list of these articles is given 
below; it will be noted that the majority of them have been published in the last 
two years. 
References; ы 

(1) Dressel, Paul L. “Тһе Effect of High School on College Grades". Journal 
of Educational Psychology, XXX (1939), pp. 612-617. 

(2) Dunlap, Jack W. Race Differences iv the Organization of Numerical and 
Verbal Abilities. Archives of Psychology, 1931, No. 124. Pp. 72. 

(3) Dunlap, Jack W. “Applications of Analysis of Variance to Educational 
Problems". Journal of Educational Research, XXXII (1940), pp. 434-442. 

(4) Lev, Joseph. “Evaluation of Test Items by the Method of Analysis of 
Variance". Journal of Educational Psychology, ХХ1Х (1988), pp. 623-630. 

(5) Rubin-Rabson, Grace. ‘‘Studies in the Psychology of Memorizing Piano 
Music. 1, A Comparison of the Unilateral and the Coordinated Approaches’. 
Journal of Educational Psychology, XXX (1939), pp. 321-345. 
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be ascribed to certain known factors or components. It must not be 
confused with the factor-analysis methods, based ona similar principle, 
now used in psychology and education. In the method considered 
here, the. factors or components are known and specified beforehand, 
and the experiments must be carefully planned and arranged in order 
that the influence of these factors may be isolated and measured. 
We obtain, at the same time, an estimate of the experimental érror 
free from the effect of these known and measurable factors. The use 
of the degrees of freedom,? also introduced by Fisher, makes the 
method equally valid for large and small samples. 

The tests of significance of the various factors are based on the 
comparison of two estimates of variance: the variance ascribable to 
the factor under consideration and to the experimental error. The 
tests all reduce to the single one of determining whether one estimate 
of variance, with 7) degrees of freedom, is significantly greater than 
a second independent estimate of the same variance with из degrees 
of freedom. The problem involved in making this test was solved 
by Fisher(1), and the tables of 22 which he has prepared and published, 
or the tables of F: published by Snedecor(11), enable us to make the 
test in the particular cases in Which we are interested. The use of 
these tables will be explained and illustrated in the problems which 
follow. 


The term "variance" as used in this method refers to the square 
of the standard deviation, i.e. 


У = (1) 
where 1” represents the variance and с the standard deviation. The 
“covariance” is related to the correlation coefficient and the standard 
deviations of two related variables. This relation may be expressed as 


С, = раз бі өз (2) 


where С, represents the covariance, 7: and оз the standard deviations 
of the first and second variables, respectively, and рі; represents the 
correlation coefficient. In the analysis of variance, therefore, we 
consider the variation of only one quantity, while in the analysis of 
covariance we consider the associated Variation, or the covariation, 


2Тһе term “degrees of freedom" refers to the number used as divisor in our 
estimate of the variance. The bias of our estimate in small samples is compensated, 
on the average, by using the number of degrees of freedom instead of the number 
of observations. (Sce equation (13).) 

?Tables VI and VII, pp. 230-253. (See Appendix A.) 

*Table 10.3, pages 184-187, (See Appendix B.) 
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of two quantities. It will readily be seen that the analysis of co- 
variance is a natural extension of the general analysis of variance 
method. 

In estimating the variance, V, we first calculate the ‘‘sum of 
squares”, as it is known, which is simply an abbreviated phrase 
denoting the sum of squares of the deviations of a set of observed 
values from their mean. It is, actually, the total sum of squares 
rather than the variance which is broken up into parts in the analysis. 
The additive properties, to be discussed later, are common to both 
the sum of squares and the degrees of freedom, and not, strictly 
speaking, to the estimates of the variance. The general term “теап 
square" is used to denote the various estimates of variance which 
are calculated; it is, as the name suggests, the quotient of a sum of 
squares and the appropriate number of degrees of freedom. If we 
denote by X any value of a variable, by X the mean value of X as calcu- 
lated from a sample, by 2) summation or addition, and by f the 
corresponding number of degrees of freedom, then the sum of squares 
may be written* > 

a(x —X)? (8) 

and the mean square 
1 
f 


The statistical theory underlying the method and the tests of 
significance will not be re-developed here as it would be of little 
interest to the practical research worker. It is suggested that readers 
who are interested in this side of the problem should refer to the 
texts of Fisher, and Snedecor (1, 10, 11) and in particular to inter- 
esting papers by Kolodziejczyk (4) and Johnson and Neyman (3) on 
this general problem. At the sarae time, however, it is felt that even 
the practical research worker will benefit from a study of the general 
principles underlying the theory since this knowledge will aid him 
in deciding whether or not the method is applicable to his particular 
problem, and will also help him in learning how to use it. In addition, 
it will enable him to attack new problems with more confidence and 
guide him in planning experiments designed to solve them. The 
next few pages, therefore, will be devoted to a general outline of the 
theory and of the procedure to be followed in attacking such problems. 
This section may be omitted by the non-mathematical reader as 
many of the problems encountered will be sufficiently similar to those 


xu -Xy (4) 


5In the notation of Fisher, this sum of squares is written S(X—X), and in the 
notation of Snedecor it is written ZX*-(EX)My ог ХХ? (ZX)!/z. 
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discussed later that they moy be solved without a detailed study of 
the theory. 
Ín solving a statistical problem, the statistician must first set up 
a statistical model to which the problem conforms with sufficient 
accuracy for results of practical value to be deduced. The model to 
which problems seem to conform in this particular type of analysis 
is one in which a particular observation, X;, is assumed to consist of 
а random element, 2;, plus a linear function, ш, of a specified number 
of constants or parameters. If we denote the parameters by 01, 03, .... 
сәз бұ then we may express these assumptions in mathematical form. 
Thus 
uj 7 С: +С: 6+... +С 6, (5) 


where the quantities Су, С, ..., Са, are known values—generally 
0,41, ог—1—дећегпипед by the nature of the problem under con- 
sideration. From (5), we may write 


Х=ш-% (@=1,2,...., М) 
= С: + .... Сы 0 bz; (6) 
It 15 assumed, further, that these random elements, in successive 
observations, are independent of each other and of the values of the 


parameters, and are normally distributed about zero, with the same 
standard deviation c in all cases, i.e. 


bs) = eee т/а? (7) 


where 2(2,) denotes the probability distribution of z; From equation 
(6), it follows that 


Ре) = 2-е = (8) 


то 


and the probability distribution of all the z's will be 


Para vm): 22: (9) 


In equations (7), (8), (9), с is the standard deviation of 2% in the 
population from which we are sampling. It is not necessary to know 
‘Parameters are constants describing the properties of the population from 
which we are sampling. 
12 


its precise value as the tests of significance are independent of the 
particular value of о. In educational problems the estimate of c is 
very useful, but this is a different question, statistically speaking. 

The value of с, as we see from the above assumptions, is supposed 
to be the same whatever the value of the parameters 01, 02, .......- ‚бз 
Methods have been developed which may be used when this assump- 
tion is not satisfied but these will not be considered here. Experience 
shows that most of the problems conform to the above model with 
sufficient accuracy for our purpose. 

Up to this point, we have defined the model only in a general 
form; these fundamental assumptions and conditions are common to 
all the problems. In a given situation, i.e. when we are faced with 
some practical problem, the model will have to be adjusted to meet 
the specific conditions which govern it. Various hypotheses regarding 
the factors which contribute to the variation in X are conceivable, 
and we must first determine the minimum number, say s, of para- 
meters @ that are needed to define exactly the set of admissible 
hypotheses. The precise values of the parameters are unknown, but 
the hypotheses will specify either certain values of the parameters 
(zero, for example) or their relations to one another. A particular 
hypothesis will be defined by one or more relations, in general say 7 
relations where r<s, between the 5 parameters б), 6, ...... ‚ 6. The 
data collected, consisting of N observations Xi, Хо, ...... у, may then 
be used to test this hypothesis. 

Before discussing the method of developing such a test, let us 
digress for a moment to consider more exactly what we mean by the 
phrase “testing a hypothesis". Тһе set of admissible hypotheses 
consists of all possible hypotheses regarding the unknown values of 
the parameters or their relations to one another which are relevant 
to our problem, or, as is more feequently the case, which we admit 
to be relevant. The particular hypothesis which we wish to test will 
be defined by the 7 relations between the parameters. Within the 
set of admissible hypotheses there will be others which are alternative 
to this one, i.e. which are contradictory to it. Let us denote by Hy 
the hypothesis to be tested’, by 27 some other hypothesis contra- 
dictory to Ho, by W the sample space, and by w any region in this 
sample space. The coordinates of the sample space, W, are the 


7In the examples which follow there is frequently more than one hypothesis 
to be tested; these have been denoted by Ho, Hi, Hz, Ha, ...., etc. Since the problems 
considered in the different examples are not the same, the symbols, Ho for example, 
may refer to different hypotheses in different examples, In each case, of course, 
the hypothesis to be tested has been defined and discussed. 
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possible results of experiment; and the observed results in any par- 
ticular experiment will define a point, Е, called the sample point, in 
W. Denote by p{Eew} the probability that Е, as determined by 
some experimental results, will fall within the region w, and by 
p\Eew/II} the probability of E falling within w as determined by 
some hypothesis, H. 

Any test of a statistical hypothesis, Hy, reduces to the rule of 
rejecting Hy when the sample point, Е, falls within some specified 
region, wo, called critical, and of accepting ZZ, (or at least not rejecting 
it) in all other cases. There are two kinds of errors which may arise 
in testing statistical hypotheses: (1) we may reject the hypothesis 
tested, Hy, when it is true, and (2) we may fail to reject Ho when 
it is false, i.e. when some alternative hypothesis, И\, is, in fact, true. 
The probability of the first kind of error is pL Eewo/Ho} and the 
probability of the second kind of error is bVEV-w)/H!| where 
W — 100 denotes the part of W outside of wo. 

The theory of testing statistical hypotheses as developed by 
Neyman and Pearson (6, 7. 8. 9) is based on the simple concept of 
arranging the test, i.e. of choosing the critical region wo, so as to 
minimize the probability of errors. The critical regions used in the 
tests of the statistical hypotheses in this bulletin appear to be the 
best possible in the sense that, (1) the probability of errors of the 
first kind is controlled at a fixed level, and (2) the probability of 
errors of the second kind seems to be reduced to as low a level as 
possible for the class of hypotheses in which we are interested. 

Our problem of developing a test of a particular hypothesis 
reduces, therefore, to that of determining a critical region 70 possessing 
the above properties. The appropriate region in our case is determined 
by a comparison of sums of squares; these are obtained as follows: 


Denote by 


S 
= У [X;—m;}? (10) 
where ж 


mi= Си Сф ..... + Cg; (11) 


is similar to uj of (5) except that the unknown parameters 0 are 
replaced by continuous variables q. 


Step 1. 


Minimize x° with regard to the s unknown q's and denote the 
resulting absolute minimum by x}. This consists in obtaining the 
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derivatives of x? with regard to all the q's, equating these to zero, 
and solving the resulting equations for the values of g;, say g, which 
minimize ха. We obtain the required minimum value of x^, ха, by 
substituting q? in (10). 


Step 2. 

Minimize x? subject to the condition that the s variables д; satisfy 
the r relations specified by the hypothesis tested. Proceeding as in 
Step 1, we obtain the required relative minimum value of x^, denoted 

2 


by ху. 


Step 3. 
Calculate the quantity 


2=1 los | 55 / х] (12) 
1 2 


where f; and f» denote degrees of freedom, defined as follows: 


h-r 
РМ (13) 


Step 4. 
Using this quantity’, refer to Fisher's tables of 2 (these are entered 
. with the above degrees of freedom which Fisher denotes by п, and 
по, respectively) and 
(a) reject the hypothesis tested, Jo, if the calculated value of 
is greater than the 5% (or 1%) point given in the tables, 
or (b) accept the hypothesis tested, Но, if the calculated value of 
is less than the 5% (or 1%) point given in the tables. 

The question of whether to две the 5%, 1%, or 0.196 point as 
the boundary of the critical region is a personal one, and depends 
on what probability of the first kind of error we consider as per- 
missible. The general practice seems to be to reject the hypothesis 
tested if 2 is greater than, i.e. lies beyond, the 1% point, to remain 
in doubt if it lies between the 5% and 1°% points, and to accept the 
hypothesis tested if s is less than the 5% point. It is impossible to 
give a definite ruling because in many cases the particular level to 
be used »vill be determined by the nature of the problem under 
consideration. 


о 


о 


ЗЕ Snedecor's tables of Е are to be used, we calculate F= сап J = in- 
л 
stead of 2. The procedure explained below із applicable to both cases. 
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Notes on the use of Fisher's tables of z. 
(1) Example (Artificial) 


Let us assume that our experimental results give us the following 
values for the test of a statistical hypothesis, Ho. 


fi=r=6 
f2=N—s=60 (14) 
2=0.8 


We proceed as follows: from Table VI, pages 250, 251 (1), or Appendix 
A, we find from the column headed пл =6 and the row headed 7-60, 
that the 5% point of the distribution of z is 0.4064; from Table VI, 
pages 252, 253, (1) we find from the column headed z;—6 and the 
row headed и: =60, that the 1% point of the distribution of z is 
0.5687. Our calculated value of z, 2=0.8, is greater than the 1% 
point so our sample point, E, falls within the critical region and we 
reject the hypothesis to be tested, Ho. 


(2) The tables are to be entered with degrees of freedom 71 cor- 
responding to the larger mean square. In the event of 


2.2 2 
Xr Ха < Ха (15) 


r N-s 
2 2_ 2 
ential, A] 


and enter the tables with degrees of freedom, 


we calculate 


ni-N-—s (17) 
"i-r 


Otherwise the procedure is the same as that outlined above in the 
first example. The interpretation of the results, however, is some- 
times difficult in the case of the rejection of the hypothesis to be 
tested, Мо. The clue to the interpretation in these cases can generally 


be found by a close examination of the nature of the hypotheses 
alternative to the one tested. 


(3) The 5% and 1% points of the distribution of z for values of 
nı and ж» not appearing in the tables may be obtained by simple 
linear interpolation. The following examples illustrate the procedure 
to be followed in such cases: 

(a) Assume that we have 71-16 and 7-60 and wish to find the 
5% point of the distribution of z; the nearest tabled values are for 


16 


пл =12, n,—24 and z;—60. From Table VI, pages 250 and 251, or 
Appendix A, we find 


To find the value of the 5% point corresponding to пл =16, we inter- 


polate in the above table with values al instead of т, i.e. 


71 
ЕЛЕЛЕЕ 
ЕЛЕЛЕН 
== 


to obtain the desired value, 0:295. 


(b) Assume that we have 7) =20 and 2. —120 and wish to find the 
5% point of the distribution of z; the nearest tabled values are for 
пл =12, 11224, по =60 and из =>. From Table VI, pages 250 and 
251, or Appendix A, we find 


In this case we must perform a double interpolation, in either 


24 60 
one of two ways, so we take — and — and proceed as before. We 
пу по 


obtain 


0.287 


|» | = | 0.2804 0.223 0.2085 


The value so obtained, 0.25, is accurate to the second decimal place, 
which is sufficient for our purpose, as an approximate value of the 
probability of errors of the first kind is all we require. If we find, 
for example, that the value of z calculated from experimental results 
falls nearly at the 1% point, we know that in repeated sampling we 
should obtain a value as great or greater than this by chance alone 
in about 1% of such cases. A knowledge of the exact proportion, or 
the exact value of the 1% point currect to 4 decimal places, is not 
necessary here as we should be inclined to reject the hypothesis in 
any case. 

The remainder of this bulletin is devoted to a study of the theory 
etc., as applied to particular problems. As this method has seldom 
been used by others in the fie!d of education, most of these problems 
are ones in which we have been particularly interested. It is hoped 
that enough of the field will be covered to enable other workers to 
apply similar methods in the solution of their own problems. 


CHAPTER II 
ANALYSIS OF VARIANCE 


In this chapter we shal! discuss the application of the Analysis of 
Variance method to certain educational problems and to the inter- 
pretation of the results. The examples considered are simple, but 
they will illustrate the use of the method. More complex problems 
will occur, of course, but it is suggested that the solution of these will 
not be difficult if the fundamental principles outlined in the first chapter 
and applied here are clearly understood. 

It is convenient to divide the examples of this chapter into two 
parts: 

(1) examples in which there are equal numbers of observations in 

the classes, and 

(2) examples in which there are unequal numbers of observations 

in the classes. 


The general method to be used is the same for both types, but the 
calculation òf the results is simpler for the first. 


PART 1 


Equal Numbers of Observations in the Classes 


Example 1. Resemblance of Identical Twins in Intelligence. 


This example is based on data collected by Wingfield (14) and 
refers to the problem of measuring the resemblance of identical twins 
in general intelligence. The data are given in Table I; columns 2 and 
3 give the composite mental age scores of 31 identical (girl) twin pairs. 

The only factors which enter here are, (1) the mental age of the 
groups tested, (2) the mental age of each twin pair, and (3) the differ- 
ence between the mental ages of the members of each twin pair. There 
is one other factor, the error involved in the measurement of mental 
age, but it cannot be considered at present as the data are not arranged 
to give us a separate measure of the influence of this factor. This 
particular problem will be considered later (see Example 5); it is suffi- 
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TABLE 1 


MENTAL AGEs OF IDENTICAL GIRL Twins 


Twin Mental Age Sum Diff. 
Pair Tou mei Ы 
(0) Хи X» Хи+Ха | Хи—Ха | 
ств: 178 174 352 4 
2 111 117 228 6 
3 102 123 225 21 
4 119 12+ 243 5 
5 137 . 130 267 7 
6 132 135 267 3 
7 136 141 277 5 
8 126 123 249 3 
9 188 159 347 29 
10 136 140 276 4 
11 137 155 292 18 
12 124 118 242 6 
13 99 104 203 5 
14 140 137 277 3 
15 166 180 346 14 
16 112 121 233 9 
17 117 130 247 13 
18 136 148 284 12 
19 143 149 292 6 
20 121 122 243 1 
21 184 187 371 3 
22 153 167 320 14 
23 135 126 261 9 
24 218 191 409 27 
25 156 162 318 6 
26 145 121 266 24 
27 165 158 323 7 
28 167 186 353 19 
29 127 117 244 10 
30 137 142 279 5 
31 143 154 297 11 
Sum 4,390 4,441 8,831 =: 
Sum of Squares, 642,732 653,179 2,587,051 4,771 
н | 
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ча, 
= #Жыз өс” 


cient to note here that part of the difference between the mental ages 
of the members of each twin pair may be due to this factor. 

We may express the relationship between the factors to be con- 
sidered as follows: Denote by 


Хи=А-+ C 2; (18) 
the mental age of the i-th member of the ¢-th pair of twins, where 
#=1,2;:=1,2,....,п. А іѕа теаѕиге of the common mental age of 


the group of girls tested, and is defined as the arithmetic mean of the 
mental ages for all individuals; C, is a measure of the mental age of the 
t-th twin pair; 2; is a measure of the differences between the mental 
ages of each twin pair. Since A is considered as the common mental 
age of the group tested and defined as the mean mental age for all 
individuals, it is necessary that 


УС=0 (19) 
t 


. 


The hypothesis we wish (о test is 
Но: С,=0 ; і-1,2,....,п-1 (20) 
i.e. the hypothesis that the mental age of an individual is independent 
of the particular twin pair to which she belongs. This is equivalent, 
as Fisher has shown (1, section 40, page 228), to the hypothesis that 
the correlation between twins (intraclass correlation) is zero. 
Following the general method outlined in the first section, we first 


write x-XX (X46): (21) 
Minimizing x^ with regard to A and C,, we obtain 

дее 2 XXX, EU (22) 

G= важи —Х.=х.-х. (23) 


Substituting these values in (21) to obtain the absolute minimum value 
of x?, we have 2 x = ke 
* x; У(Х.) (24) 
$c o 
If the hypothesis to be tested, №: C, —0, is true, then (21) becomes 
x!-XXGt—A)»y (25) 
та 

, Minimizing with regard to A, and substituting the obtained value 

A =Х.. in (25) we obtain the relative minimum 


з= (Ха X.) 
S,C.E.R.T., West Benga) ы e ) 


Date 1-53... 21 


Acc, A ce ВИ "m 3 No... 6e 


We may rewrite (26) in the. form т 
х= УУХ, -—XJHEXXGG -Х.)® 
Pa i 

=x +x ‚ say. (27) 

The degrees of freedom are 
fı=r=n—1 \ 
fo=N-—s=2n—n=n (28) 
For purposes of calculation it is simpler to write x2 and x? in the 
form 


xa 35 (Хи Ха)? (29) 
(Xx; 
ж=} Ut Xa)? — EE cu (30) 
We may check the calculations by obtaining Р 
(их) 
xP = DXF, -i (31) 
за 2n 


separately, and using the identity 
Xr = ха + xb (32) 
The easiest way to calculate the necessary values is that shown in 
Table I; we form the sum and difference for each pair of values and 
then calculate the sum and sum of squares for each column, except the 
last where only the sum of squares is required. This method has the 


added advantage of enabling us to check the calculationsat each stage. 
From the last two rows of Table I, we find 


Z(X,—X3)! = 4,771 

xa u- X3)? = 2,587,051 

| ZXX,- 8,831 
xxx, = 1,295,911 


Substituting these values in (29), (30), (31), we have 

xi = 35,677.7419 

х= 2,385.5000 

x? = 38,063.2419 

It is convenient, and also customary, to place all these values in 

one table. This has been done in Table II, which also shows the 
notation generally used in the analysis of variance; the column headed 
"d.f." refers to the degrees of freedom associated with the sum of 
squares given in the third column. 


22 


TABLE П 
ANALYSIS OF VARIANCE OF MENTAL AGEs OF IDENTICAL GIRL TWINS 


Variance d.f. Sum of Squares Mean Square 
Between Pairs . . . . 30 35,677.7419 1,189.2581 
Within Pairs. . . . . 31 2,385.5000 76.9516 
Total а 61 38,063.2419 — 


The additive property, mentioned in the first section, appears very 
clearly here: the degree of freedom and the sum of squares in the first 
two rows (for between and within pairs of twins) add up to the total 
shown in the last row. 

There remains only the test of the hypothesis Но. For this we 


calculate SETS (am) 
8-1 76.9516 


=} log, {15.4546} =1:369 


and refer to Fisher's tables of z with degrees of freedom f; =”, —30 and 
=пз=31. The value is clearly greater than the 1% point, so, follow- 
ing our rule, we reject the hypothesis to be tested. This means that 
the mental age of an individual is not independent of the twin pair to 
which she belongs; or the intraclass correlation between twins is greater 
than zero. An unbiased estimate of the intraclass correlation, r, is 
obtained by solving the equation (see Fisher (1), page 231): 
l--r  1189.2581 


==—== = 15.4546 
1—r • 76.9516 


In this case we find r=0.88. 

The results of a similar analysés of the chronological age of the same 
group of twins are shown in Table III; in this case, of course, the differ- 
ences between the members of each twin pair are zero, so the within 
pairs sum of squares and mean square are zero. 


TABLE III 


ANALYSIS OF VARIANCE OF CHRONOLOGICAL AGES OF 
IDENTICAL GIRL Twins 


Variance d.f. Sum of Squares. Mean Square 
Between Pairs . . . . = 30 27,871.4839 929.0495 
Within Pairs af Reet с 31 0 0 
ога 3 4% She A ys 61 27,871.4839 -- 
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A comparison of Tables II and III shows very clearly how, in the 
analysis of variance, the total variation is broken up into parts which 
may be ascribed to different factors. 


Example 2. Reliability of Tests. 

The following examples refer to the problem of determining the 
reliability of а test,’ in this case an achievement test in French Reading 
for Grade X. As a discussion of the theory underlying the analysis 
may be found in my paper (2), only a general outline will be given here. 

The factors in which we are interested are the common ability of 
the group tested, the ability of each individual, the trial or practice 
effect and the errors of measurement. Let us assume that the score 
Хи of the t-th individual on the s-th trial of the test may be repre- 
sented as a sum of these factors or components. We may express this 
in a mathematical form by assuming that 

Хи=А +В, Cz (32) 
wheres=1,2; #=1,2,....,п. Aisa measure of the common ability 
of the group tested and is defined as the mean for all trials and indi- 
viduals; B, is а measure of the trial ог practice effect; С, is a measure 
of the ability of the ¢-th individual; 24 represents the errors of meas- 
urement. Finally, since A is defined as the mean for all trials and 
individuals, it is necessary that 

ХВ,-0 " 
52 =0 | (33) 


The hypotheses which we wish to test are, (1) the hypothesis 271, 
that the trial or practice effect is Zero, i.e. 


Hy: B,:0 (34) 
and (2) the hypothesis, И», that the individual effect is zero, i.e. 
По: C, 20 (35) 


The latter test will tell us whether or not 
measures the abilities of the individuals. 


To develop the necessary tests of these hypotheses, we first write 
х-ХХ(Х,-А-В, ma (36) 
st 


the mental test actually 


and minimize with regard to all the quantities A, В,, C, to obtain the 


"The word "test" is used below in two distinctly different Senses; it may refer 
to the achievement test or to the test of a hypothesis. There should be no confusion, 
however, as the meaning is in every case made clear by the context, 
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absolute minimum, x7. Assuming that JJ, and H, are true, separately, 
we minimize with regard to the remaining quantities to obtain the 
relative minimum values of x^, say ха and ха, required in the test of 
H, and Hp, respectively. We find 


xi-XXQG -X, Kat. у (37) 
ха = +, val ag) 
xà xix» say’ 

ques xi -ZXOt, -X * (39) 
xi-XXOG-X Ју 40) 
and Xa =} KG 
x,- УХ, (41) 
"i 
1 Е 
ioe 


The additive property of the sum of squares is demonstrated in the 
identit ” y 19 2 2 2 
d EXQG,-X.)exbtxbt (42) 
pg 


which may also be used, as shown in the analysis of variance table, to 
check the calculations. 
For purposes of calculation,’ equations (37), (39), (40) may be 


written as E 


2 2 2 (УХА) 
x; УХ — Lg УХ (13) 
st ns tos 2n 
у. хх) 
xi ~ (УХ) (44) 
"ms t a 
Ух. n 
х- 0х.) — ? (45) 
= 
or the first two may be written 
ar EO Y Д > S 
x;—-àMOXu-— Хи) — — (Zcu-X9r (46) 
t 2n т 


= = EG хау (47) 


25 


and 


УЖА.) = БУХА — = (ххх, } (48) 
Ж 2 г ЫМ st 


The general method used in calculating the sums of squares can be 
seen if we examine the items in equations (43), (44), (45) and (48). We 
use sums, sums of squares and sums of squared sums, i.e. 

ххх = 
ХХ, and Xx 
t $ 
ХХ, and D(LX,,)? (49) 
5 4 5 
ХХХ, and (DD X,,)? 
54 st 


or, in simple cases like the present one, sums, squares of sums, differ- 
ences and squares of differences as in equations (45), (46) and (47). 


To test the hypothesis Hi, we calculate 


а= ог, М / | (50) 


and refer to Fisher's tables of z with degrees of freedom пл =ј] = 1 апа 
по = jJ. з=п-1. 


To test the hypothesis H», we calculate 


T pa 
"iios у ха) 
Б i5 fs ES 


and refer to Fisher's tables of z with degrees of freedom 
пл =m: =), =f} =n—1, 


In the following examples we have applied the method to two sets 
of results for the French Reading test mentioned in the first paragraph; 
they refer to the same test, but to results from different classes of pupils 
(both in Grade X). These particular examples were chosen because 
they illustrate nearly all the Possible kinds of results which we may 
obtain. 
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, 
Example 2 (а) 
TABLE IV 
Scores RECEIVED Bv PuPILs ON FonMs А AND B OF THE FRENCH READING TEST (а) 
Individual, Зоте on Sum Difference 
1 Form A Form B рл Жос- Хз 
Хи X» 
1 17 20 37 —3 
2 18 20 38 —2 
3 19 23 42 —4 
4 11 7 18 4 
5 19 22 41 -3 
6 16 20 36 —4 
7 36 38 74 -2 
8 9 12 21 —3 
9 12 12 24 0 
10 31 34 65 —3 
11 16 22 38 —6 
12 10 10 20 0 
18 28 30 58 —2 
14 29 38 67 -9 
15 35 34 69 1 
16 ' 14 18 32 -4 
17 28 28 56 0 
18 6 2 8 4 
19 8 12 20 -4 
20 18 23 41 —5 
21 14 20 34 —6 
22 19 e 20 39 = 
23 24 29 53 -5 
24 F 1 21 -7 
25 8 1 20 —4 
26 8 13 21 -5 
27 19 18 87 1 
28 10 9 19 1 
29 19 19 38 0 
30 24 24 ~“ 48 0 
| 31 12 14 26 -2 
| 32 25 32 57 -7 
| 33 4 5 9 -і 
34 4 9 13 -5 
| 35 22 19 41 3 
Sum 599 682 1,281 —83 
Sum of Squares| 12,797 16,198 57,461 529 


TABLE V 


ANALYSIS OF VARIANCE OF SCORES ОХ FORMS A AND В OF THE 
Екехсн READING ТЕЗТ (a) 


Variance d.f. Sum of Squares Mean Square 
S БОРИ. с wy 5 ж 1 98.4143 98.4143 
Between Individuals . . . 34 5,288.2000 155.5353 
Еке > 9, 4-5 ә 34 166.0857 4.8849 
ШАШ - a коњ > 69 5,552.7000 -- 


From the values in Table V we find, for the test of the hypo- 


thesis Hi, 98.4143 
los. _ 
4.8849 


The degrees of freedom аге л; — f, =1 and n2=f3;=34. From Fisher's 
tables we find that our value of z is greater than the 1% point, so we 
reject the hypothesis to be tested, H;. This means that in this case 
the trial or practice effect is significant. 


For the test of the hypothesis Hs, we calculate 


155.5353 
m=} osos _ di 
PST ism 


The degrees of freedom are n, => =f:=f;=34. Referring to Fisher’s 
tables of z we find that our value of 2 is beyond the 1% point, so we 
reject the hypothesis Ш». We conclude that the test measures the 
abilities of the individuals tested, or measures them 
to enable us to distinguish between the individuals. 


1502 


21= 


ын 


accurately enough 


Example 2(b). 


The original data for this example are given in Table VI and the 
results of the analysis in Table VII. There are very few cases, but 
we may apply the method as the use of degrees of freedom makes it 
equally valid for large or small ‘samples. 

In the test of the hypothesis 17), we have one of the cases discussed 
in the second note of the first chapter. We calculate 


11 Un 
5: => log, 


= » =0.4 
EA " 


and refer to Fisher's tables of 2 with degrees of freedom пл =] =17 and 
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Scores RECEIVED BY Pupits ON Forms A AND В OF 


TABLE VI 


Евехсн Reapine ТЕЗТ (b) 


THE 


Score on 
Individual Sum Difference 
t Form A Form B Хи + Хы Хи— Хи 
Хи Хә 
1 23 32 55 -9 
2 84 32 66 2 
3 33 34 67 -1 
4 81 25 56 6 
5 22 29 51 -7 
6 34 36 70 -2 
7 35 37 72 -2 
8 26 25 51 1 
9 33 28 61 5 
10 29 33 62 -4 
11 35 26 61 9 
12 26 21 47 5 
13 28 28 56 0 
14 34 35 69 -1 
15 33 29 62 4 
16 29 21 50 8 
17 28 32 60 —4 
18 27 24 51 3 
Sum 540 | 527 1,067 13 
Sum of Squares} 16,490 15,841 64,229 433 
e 
TABLE VII 


ANALYSIS OF VARIANCE OF SCORES ON FonMs A AND В or THE 
FRENCH READING ТЕЗТ (b) 


Variance d.f. ? Sum of Squares Mean Squares 
Between Forms . 1 4.6944 4.6044 Д 
Between Individuals 17 489.8056 28.5121 
Error. 17 211.8056 12.4621 
Total 35 700.3056 Judd and 
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по=ј=1. The value of z is less than the 5% point, so we accept Hi, 

ie., we conclude that the practice or trial effect is zero. We would, 

as a matter of fact, always accept the hypothesis №, in such cases, as 

попе of the hypotheses alternative to H is acceptable in this situation. 
For the test of the hypothesis He, we calculate 


p см. jana 
273 ОШ То 160] 


and refer to Fisher's tables of z with degrees of freedom лу=тп»=17. 
Our value of z is approximately equal to the 5% point, so we should 
be inclined to accept the hypothesis Н». A comparison of the results 
with those of Example 2(a) shows that in this case the errors of meas- 
urement are larger and the differences between individuals consider- 
ably smaller. A glance at the scores in Table VI will show that this 
does seem to be a selected class of students, which would explain one 
difference, but it is equally obvious that our test may not measure 
with equal accuracy in all situations. 


| ол 


Supplementary Notes on Theory. 

(1) It should be noted that the test of the hypothesis H, in cases 
such as these is equivalent to the test of significance of the difference 
between the mean scores on the A and B forms, i.e. the /-test, as it is 
known. This test could be made separately, of course, but it is con- 
venient to include it as one of the set of tests in the analysis of 
variance. The equivalence can be seen more clearly if we compare 
the figures in the first column (71 =1) of the tables of z with the cor- 
responding values given in the table of /; the relationship is 2 —log,(). 

(2) We may obtain an estimate of the probable error of an indi- 
vidual score from the mean square ascribable to "error" by taking the 
square root and multiplying by the factor, 0.6745. 

(3) The test of the hypothesis Z7, may easily be shown to be equi- 


valent to the test of the hypothesis that the reliability coefficient, 
TAB» iS Zero. 


Example 3. Errors of Marking. 


The data given in the folloving table refer to a problem we encoun- 
tered in the construction of a test. The marking of one section of the 
test was not objective and the question arose as to Whether the error 
introduced was large enough to justify the trouble of eliminating it by 
changing the form of the test or by constructing an elaborate key. 

In order to determine the magnitude of the error, we chose 20 
papers at random from those returned to us by a teacher and had these 
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marked independently by three examiners, whom we may denote by 
P,Q, В. The marks are shown in Table VIII, in the columns headed 
“Marks assigned by Examiners P, Q, R.” What factors will determine 
the mark given by an examiner to the paper of a particular individual? 
The most important, obviously, should be the true value of the paper 
(presumably a measure of the ability of the individual), but super- 
imposed on this there will be another factor due to the differences in 
the degree of severity of marking of the different examiners. Finally, 
there will be random differences, not ascribable to the above factors, 
due to the varying reactions of the examiners to different answers. 


TABLE VIII 
MARKS ASSIGNED BY EXAMINERS 
Marks assigned by Examiners 
Individual Sum 
і = Q R ZXa 
Xu X» Хз 4 

1 18 19 19 56 

2 20 20 20 60 

3 30 33 31 94 

4 17 18 19 54 

5 24 23 22 69 

6 33 32 36 101 

7 21 23 22 66 

8 16 18 18 52 

9 33 34 35 102 
10 24 o 25 26 75 
il 12 13 14 39 
12 16 © 15 16 47 
13 23 25 24 72 
14 25 27 28 80 
15 30 31 32 93 
16 22 25 25 72 
17 24 23 0 28 75 
18 27 29 27 83 
19 27 28 31 86 
20 31 30 31 92 
Sum 473 401 504 1,468 
Sum of Squares 11,873 12,749 13,468 114,060 


If we denote by X,, the mark assigned by the s-th examiner to the 

paper of the /-th individual, we may write 
Хи=А+В,-С + (52) 
where 5=1,2,8,; £—1,2,...., п=20. А isa measure of the com- 
mon ability of the group tested; it is defined as the mean of the marks 
for all examiners and individuals; B, is a measure of the differences 
between examiners; C, a measure of the ability of the /-th individual, 
and z, represents the random errors of marking. Since we have de- 
fined A as the mean of the marks for all examiners and individuals, it 

` 18 necessary that 


ZB, =0 

ус -— (53) 
The hypotheses we wish to - are 

EN со 


If the hypothesis Z is true, then our problem reduces to that of 
eliminating, if possible, the random errors of marking. If, on the 
other hand, H, is false, we have two problems, but the solution of our 
original problem is simplified in either case as we shall know the kind 
of errors which affect the results. 

If the hypothesis Hz is true, then we know that the mark assigned 
to a paper bears little relation to the ability of the individual (or the 
true value of his paper). We knew, however, before starting this 
problem, that our test was a fairly accurate measuring instrument 
(from the results of analyses similar to that considered in Example 2), 
but the test of this hypothesis is included here because it is part of the 
general analysis and will be important in other cases. 

The method used in developing the test of these hypotheses is the 
same as in the previous problems. We write 

x -ZXQt,-4-B,-cy (55) 
5 
and minimize x? 


(1) with regard to all the quantities, to obtain the 


а absolute minimum 
value of X^, x3; 


(2) with regard to the quantities remaining when we assume Н, is true, 
to obtain the first relative minimum value of x?, x2 


ZU 
(3) with regard to the quantities remaining when we assume Hiis true, 
to obtain the second relative minimum value of x?, x3. 


We obtain, finally, 
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A-lXxxx,-X 

8n s: 
Bie LEX,Q-X.-X.-X. (56) 
Calyx, -X.2X.-X. 

ӛз 


яж Вых Рух, (xxx, (эт) 
51 nr OF EY bn зт 


ха =УУ(Х, -X.y 
низах." s (Ex 


=xi+xi, say. (58) 
xà -XXQG =X.) 


= pu 31 А 2 
XX [хи] 


=xa хз, say. (59) 
To test the hypothesis Hi, we calculate 


o 


and refer to Fisher's tables of z with degrees of freedom 
1; —f1—2, and т -Уҙ-2(п-1) =38. 
To test the hypothesis H2, we calculate 
xi à 
а-не49; aj (61) 
and refer to Fisher's tables of z with degrees of freedom 
ni-fi-n—l =19, and m2=f3=2(n—1) =38. 


The complete analysis is shown in Table IX. It will be seen that 
about 3% of the total variation is ascribable to differences between 
examiners and random errors of marking. 
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TABLE IX 


ANALYSIS OF VARIANCE OF THE MARKS ASSIGNED BY THE 
DIFFERENT EXAMINERS 


Variance d.f. Sum of Squares Mean Square 
Between Examiners . . . 2 24.2333 12.1167 
Between Individuals . . . 19 2,102.9333 110.6807 
Errors of Marking. . . . 38 45.7667 1.2044 
ШІ 2 X. x miu a 59 2,172.9333 — 
12.1167 
z=} lo; { |ы 
* 056112044 
x 
2: =+ log, 4—— — > =2.260 
ame ed 1.2044 


The values of both z; and 2» are significant, i.e. greater than the 
1% points given in the tables of 2, so we should reject both H, and Ho. 
We conclude that the mark is determined mainly by the ability of the 
individual (i.e. by the true value of the paper), but that part of the 
error in marking lies in the differing degrees of severity with which the 
examiners mark the papers. It was decided to construct the key in 
such a form as would eliminate the differences between examiners and 
also reduce the random errors as much as possible. These errors were 


not large enough, however, to make it necessary to change the form of 
the test itself. 


Example 4. Tests of Hom 
from Different Experiments. 


One of the problems frequentl 


ogeneity and the Combination of Results 


| obl y encountered in experimental work 
is that of determining the condition under which it is permissible to 


combine the results from Several sub-samples. For results such as we 
are considering, i.e. for problems in which the analysis of variance 
method may be used, it is possible todo this. All the tests used in this 
method are based on the comparison of sums of squares or mean 
Squares. Consequently, the conditions to be satisfied are simply that 
the estimates of variance ascribable to particular factors in each of the 
different samples, should be estimates of a variance common to all the 


Ssumption is basic to the 
‘Oncept. We merely test 
Or groups which, because 
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of the nature of the experiment, may be taken as homogeneous in them- 
selves. The problem, in a slightly different form, was first discussed 
by Neyman and Pearson (9) and investigated further by P. P. N. 
Nayer (5) and B. L. Welch (12, 13). Exact statistical tests are not 
available for cases in which there are more than 2 sub-samples, but 
those which are suggested by the above authors appear to be adequate. 
The nature of the problem can be seen more clearly by considering 
an example. Wingfield (14) gives data relating to the measurement 
of the mental age of 57 pairs of fraternal twins; these may be of like- or 
unlike-sex, and in the like-sex pairs there are, of course, boy twins and 
girl twins. We have, therefore, three groups of fraternal twins and 
the question arises whether the results are the same for each group, i.e. 
whether we can speak of a common fraternal twin group and consider 
these three sub-groups as samples from a common population of fra- 
ternal twin pairs. The problem is not so simple, as there are several 
related problems, but this is its outline. 
Tables X, XI, XII and XIII give the original data and the results 
of the analysis for the two groups of like-sex fraternal twins; girl twins 
in Tables X and XI, and boy twins in Tables XII and XIII. The 


analysis here is the same as in Example 1. 
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TABLE X 
MENTAL-AGES OF FRATERNAL LIKE-SEX TWINs—GIRLS 


Twin Mental Age Sum Difference 
Pairs ---- Хи+Ха | Хи-Ха | 

t Хи Ха 

1 187 113 250 24 

2 173 147 320 26 

3 119 129 248 10 

4 138 150 288 12 

5 142 104 246 38 

6 191 172 363 19 

7 144 188 332 44 

8 136 113 249 23 

9 141 143 284 2 

10 141 118 259 к 23 

п 112 121 233 9 

12 147 139 286 8 

13 156 124 280 32 

14 118 155 273 37 

15 141 140 281 1 

16 108 118 221 15 

17 111 161 272 50 

18 144 132 276 12 

19 113 108 “ 221 5 

Sum 2,607 2,575 5,182 = 

Sum of Squares| 366,271 358,301 1,437,412 11,732 

TABLE XI 


ANALYSIS OF VARIANCE ОР MENTAL AGES ОР FRATERNAL 
LikE-SEex Twins—Girts 


Variance d.t. Sum of Squares Mean Square 
Between Pairs |. . . . . 18 12,044.8421 Е 669.1579 
Within Рай, 205... 19 5,866.0000 308.7368 
Таш x n 37 17,910.8421 
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TABLE XII 
MENTAL AGES OF FRATERNAL LiKE-SEx Twins—Boys 


Twin Mental Age Sum Difference 
Pairs = Хи+Ха | Хи-Ха | 
t Xu Хэ; 
1 156 176 332 20 
2 101 119 220 18 
3 108 121 229 13 
4 134 110 244 24 
5 165 174 339 9 
6 158 167 325 9 
Т 159 137 296 22 
8 130 150 280 20 
9 144 i 129 273 15 
10 173 147 320 26 
11 101 97 198 4 
12 113 123 236 10 
Sum 1,642 1,650 3,292 — 
Sum of Squares| 232,162 234,100 928,992 3,532 


TABLE XIII 
F MENTAL AGES OF FRATERNAL 


ANALYSIS OF VARIAN 
Twins—Boys 


Variance d.f. Sum of Squares | Mean Square 


|| 
Between Pairs . . . . - 11 12,943.3333 | 1,176.6667 
Within Pairs . . . + - 12 1,766.0000 147.1667 
= s ені 
Total s «€ s ва 23 | 14,709.3333 


Tables XIV, XV, XVI give the original data and the results of the 
analysis for the group of unlike-sex fraternal twins. There is an addi- 
tional problem here, namely that of determining whether or not there 
is a significant difference between the mental ages of the boys and girls. 
The first analysis, therefore, is similar to that considered in Example 2; 
the results are given in Table XV. 
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TABLE XIV 
MENTAL AGEs oF UNLIKE-SEX FRATERNAL TWINS 


Mentai Ages 
Pairs Sum Difference 
t Boys Girls Хи+Хә Хи-Хи 
Хи Хэ; 

1 97 по 207 —13 
2 129 103 232 26 
3 131 139 270 У 
4 151 132 283 19 
5 180 140 320 40 
6 124 118 242 6 
7 133 142 275 = 9 
8 158 160 318 = 2 
9 128 145 273 = 
10 140 145 285 -5 
п 126 117 243 9 
12 130 136 266 - 6 
13 139 205 344 —66 
14 130 111 241 ' 19 
15 113 138 251 —25 
16 140 162 302 —22 
17 177 146 323 31 
18 141 144 285 = 3 
19 134 135 ` 269 = 
20 120 127 247 = 
21 116 116 232 0 
22 138 177 315 —89 
23 122 125 247 == 
24 122 158 280 —36 
25 149 164 313 -15 
26 161 158 819 3 
Sum 3,529 3.053 7,182 —124 
Sum of Squares| 487,747 526,251 2,014,608 13,328 
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TABLE XV 
ANALYSIS OF VARIANCE OF MENTAL AGEs OF UNLIKE-SEX 
FRATERNAL TWINS 


Variance | df. | Sum of Squares Mean Square 
Between Sexes... - - | 1 | 295.6923 295.6923 
Between Pairs 2% 9 25 | 15,389.3077 615.5723 
Error (Within Pairs) Я | 25 | 6,368.3077 254.7323 
Nub „олуи | 51 | 92,053.3077 | ак ыш 


We find that this difference is not significant, 


жын! 5 
log 2905925 не 
T E 


Пі-і, па=25 


z= 
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so we may present the results in a form similar to that for the first 


two groups, as shown in Table XVI. 


TABLE XVI 
ANALYSIS OF VARIANCE OF MENTAL AGEs оғ UNLIKE-SEX 
FRATERNAL TWINS (FINAL ANALYSIS) 


Variance | df. | Sum of Squares Mean Square 
Беру сеп Байа 25 | 153893077 | 615573 — 
Мин ране „о. eo ®| 295] 6,664.0000 256.3077 
Total ТЫ! | 22,053.3077 5 


are based оп tbe assumption that we may write 

Ха= А С.а (62) 
where s=1, 2, 3; 7=1, 2; і-1,2,.....", Тһе additional sub- 
script 5 is introduced to denote the particular group of fraternal twins 
considered. 4, is a measure of the common mental age of the s-th 
group, and is defined as the arithmetic?mean of the mental ages for all 
individuals within the s-th group; Cs is а measure of the mental age of 
the /-th twin pair within (һе s-th group; 2,2154 measure of the differ- 
ences between the mental ages of each twin pair within the s-th group. 
We must, of course, have, as before, 
ХС,-0 for each s. (63) 


t 
in the first instance, that sy; is ar 
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The analyses 


andom term пог 
We assume, 1 ic 


distributed about zero with standard deviation 0,, say. If the groups 
are samples from a common fraternal twin population, then c, will be 
the same for all s; i.e. z,; will be distributed with standard deviation 
c, say, constant for all groups. The first hypothesis we wish to test, 
therefore, may be written 

Hy: с,=< (64) 


Welch (13) has suggested a test of this hypothesis similar to that 
originally suggested by Neyman and Pearson. We calculate 


"s. 
лет (2) N =) 
s NS 5 g 


"ns 


[HE Е 
а N (65) 


where N = У, т denotes the product, and f! denotes the within pairs 


s 
sum of squares for the s-th group; and refer to Nayer’s tables’? of the 
L, distribution with & —3 and degrees of freedom реја where fj 


5 
denotes the degrees of freedom associated with 0! in the s-th group. 
The rule to be followed in using these tables is to reject the hypothesis 
tested when the calculated value of L, is less than the corresponding 
1% point given in the table. 

To find the value of Ly, we first calculate the value of log Lı where 


1 
log Lı =log № — — Ум, log п, + L En, log 0; — log (20) (66) 
Ns Ns 5 ы 


and then find 1, from a table of antilogarithms. For our case, we 


have: 
TABLE XVII 
CALCULATION OF LOG L; FOR THE TEST OF THE HvrorHEsIs Hi: o,=0 
| 
Ја Ns log ns ns log ns | o! log 0} | ns log 0! 
26 5 j == 66 а 
52 | 1.7160 | 6664 | 38237 | —— 
19 38 | 1.5798 — | 586% | 3.7683 | — 
12 24 | 13802 — | ums | зат | — 
ips x wur pem 3 
57 114 | = ns log п; =185.3890 | 14,296 | Zn, log 01 =419.9590 
| pae i 
| 


We find f; =19, Lı =0.967, and from Nayer's tables we see that this is 

greater than the 5% point (0.898), so we accept the hypothesis H,- 
The other variance in which we are interested is ascribable to 

between pairs or, in the notation of equation (63), ascribable to the 


1(5), page 51, Tables IV and У. (See Appendix 6 
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factor Сы. If the С, are not all equal to zero, we may assume that 
C, is normally distributed about zero with standard deviation с, say. 
If the groups are all samples from a common population of fraternal 
twins, еп с, will be the same for all s, ie. о, =, for alls. The 
hypothesis we wish to test, therefore, may be written in the form 
Нә: с =; (67) 
We cannot test Н» directly, unfortunately, but we can test it indirectly 
if we may assume H, is true. We consider another hypothesis Hs, 
Hy: [ea s (68) 
| =o 
and if both Нз and H are true, then H: must also be true. 

The hypothesis Н; is equivalent, in this case, to the one originally 
considered by Neyman and Pearson. ИН, is true, than Хи; is nor- 
mally distributed about a mean A, with constant standard deviation; 
to test H, we calculate, following the method of Welch (13), 


те о, "s 
үү del 
„= Кох : zw 69 
, (2 Hb (69) 


where 6, denotes the total sum of squares for the s-th group; and refer 
to Nayer's tables of Ја with k=3 and degrees of freedom f=} УЛ; 


5 
where f, denotes the degrees of freedom associated with 0, in the s-th 


group. 
Proceeding as before, we first calculate log Lı as shown in the fol- 


lowing table: 
TABLE XVIII 


o = 
CALCULATION OF LOG L, FOR THE TEST OF tue Hypotuesis Hs: в 
ae 
T e | 
КА | Hs | Тоц ts n, log ns | 8. | log 0, ns log 6, 
| == E | | 220533077 | 43435 | — 
5 | | | | | 
37 | (See Table XVID | | платом | 42531 | -- 
з | | | 14,709.3333 | 4.1676 | -- 
u i ze ME Ж Хи, log 0, =487.5013 
111 | | mie с б, 7.5 


We find f —37, І,-0.990, and from Nayer's tables we see that Ла is 
greater than the 5% point, so we accept H;—and therefore li. We 
mav assume that the three groups are samples from the same common 
population of fraternal twins, and combine the results. One further 
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test in the analysis and the final combination of the results will be 
discussed in the first example of Part 2. 

It was pointed out that in the case of two samples there is an exact 
test of these hypotheses. In such cases, L, is a function of 2, S0 we 
may use Fisher’s 2 test and the tables of z in making the tests; other- 
wise the procedure is the same. The tests in this form are applied in 
the first part of the next example. 


Example 5. Differences in the Mental Ages of Identical Twins. 


In this example we return to the problem raised in the first: whether 
the differences between twins may be due solely to the errors of meas- 
uring by means of mental tests. The following data are taken from 
Wingfield (14), and refer to the results from two mental tests given to 
31 pairs of identical girl twins. From these we can find the answer to 
our question; the differences between the measurements of the same 
individuals will give us an estimate of the errors of measuring which 
we can use as a basis for our tests. 

Before proceeding with this analysis, however, it is necessary to 
analyze the data separately for each mental test in order that we may 
make certain it is permissible to combine the results. We shall use 
the results deduced in the last example and, since there are only two 
samples, apply the exact tests of significance. The original data and 
the results of the analyses are shown in the following tables; the method 
used in the analysis is similar to that discussed in Example 1. 


156 Mental Test 


TABLE XIX 
MENTAL AGES оғ IDENTICAL GIRL TWINS 


2nd Mental Test 


Mental Age Mental Age 
Twin || Sum | Diff. Twin: |————— бам | пр 
Pairs Ж y X+Y |Х—У|| | Pairs X ¥ XY |!X—Y| 
1 156 163 319 7 1 192 192 384 0 
2 120 113 233 7 2 102 121 223 19 
3 108 115 223 7 3 130 95 225 35 
4 112 120 232 8 4 126 128 254 2 
5 134 120 254 14 5 139 139 278 0 
6 131 136 267 5 6 133 133 266 0 
7 131 138 269 7 7 142 145 287 3 
8 124 126 250 2 8 122 125 247 3 
9 216 158 374 | 58 9 159 160 319 л 
10 135 136 271 1 10 137 143 280 6 
1 136 181 287 15 11 138 159 297 21 
12 121 113 234 8 12 126 123 249 3 
13 102 112 214 10 13 96 96 192 0 
14 136 133 269 3 14 144 140 284 4 
15 158: | 197 355 | 39 15 174 | 162 336 | 12 
16 114 121 235 7 16 110 120 230 10 
17 125 | 127 252 2 17 134 | 107 243 от 
18 138 144 282 6 18 134 152 286 18 
19 142 158 300 16 19 145 140 285 5 
20 120 115 235 5 20 124 126 250 2 
21 176 175 351 1 21 198 192 390 6 
22 145 143 288 2 22 160 190 350 30 
23 118 123 241 5 23 134 147 281 13° 
24 225 189 414 30 24 210 192 402 18 
25 150 163 313 13 25 161 160 321 1 
26 141 124 265 17 26 148 118 266 30 
27 164 154 318 10 27 165 162 327 3 
28 145 | 179 | 324 | 34 28 | 188 | 192 | 380 | 4 
29 126 121 247 5 29 108 132 240 24 
30 131 125 256 6 30 153 149 302 4 
31 134 156 290 | 22 31 151 151 302 0 
Sum | 4,314 | 4,348 | 8,662 | — Sum |4,483 | 4,491 | 8,974 = 
Sum of Sum of 
Squares| 622,554| 626,774 0,488,932] 9,724 Squares 670,841] 673,017/2,681,332| 6,384 
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TABLE XX 


ANALYSIS OF VARIANCE OF MENTAL AGES ОР IDENTICAL 
GirL Twins 


Sum of Squares Mean Square 
Variance df. 
Test 1 Test 2 Test 1 Test 2 
Between Pairs... .| 30 34,300.7741 | 41,751.8710 | 1,143.3591 1,391.7290 
Within Pairs...... 31 4,862.0000 | 3,192.0000 156.8387 102.9677 


ТОҒА), is serene 61 | 39,162.7741 | 44,943.8710 642.0127 736.7848 
| 


The within pairs and between pairs mean squares for the two tests are 

not significantly different, as we may show by calculating 
(1) for within pairs 

қ = 

z=} log, 4 — —- 

102.9677 

пл —75—31 

(2) for total (see test of hypothesis Из in Example 4) 

| {736.7848 

Og, 4 ——— — 

1642.0127 

пу =n2.=61 


and referring to Fisher's tables of ZW 
freedom. Since these 


| оз 


zi 
5=2 


| =0.070 


ith the appropriate degrees of 
are not significant, we may combine the results. 
The factors present in our final analysis will be as follows: 


(1) a measure of the common abi'it 
we may denote by 4 and d 
and individuals; 

(2) a measure of the abilit 
denote by С; 

(3) a measure of the differences between the members of each' 
pair of twins which we may denote by B,,; 

(4) a measure of the difference between the tests, denoted 
by D,; 

(5) the errors of measurement of mental age, denoted Бу Zstu- 


| If we denote by Xy, the score (i.e. mental age) made by the s-th 
individual of the ¢-th pair on the w-th test, we may write 


y of the group tested; this 
efine as the mean for all tests 


y of each twin pair which we may 


Xa = А+В,+С+Р0, es, (70) 
"yt i S=1, 25 uz, 25 Ny. 
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where /21,2,... 


Since we have defined A as the mean for all individuals and all tests, 
it is necessary that 


УВ, =0 
M (71) 
XB, =0 


The hypotheses we wish to test are 

ну: D,=0 (72) 
ie. the hypothesis that there is no difference between the tests; 
and Н,: C,=0 (78) 
i.e. the hypothesis that the score is independent of the particular twin 
pair to which the individual belongs; 
and На: By, =0 (74) 
ie. the hypothesis that the difference between the members of each 
twin pair is zero. 


Denote by 
x 2XXX Хи -А —B,—C,—D,}? (75) 


and minimize x? 
(1) with regard to all the quantities, to obtain the absolute 
minimum value of ж, ж; 4 59 
(2) with regard to the quantities remaining if H, is true, to 
obtain the first relative minimum value of x^, ха; 
(3) with regard to the Quantities remaining if Ha 5 true, to obtain 
the second relative minimum value of x ; хе: ; 
(4) with regard to the q@antities, remaining if На is true, to 
third relative minimum value of x’, хд. 


obtain the 
We find 
222 Ke 
1 —X -Х.,-Х.. Y 
D,7 = Da Хи X.-X X (76) 


с, = Хин Xs =Х..-Х... 


Ва= уха —iXXXwu =Х ши 
5 5 45 N К 
х= (Ха -X-u Dy 


stu 
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Ха-ха у EXC Хаа)" 


ха -ХХХ pes —X,,.}? 
met + {DE x4)! 


=x2+xi, say, 
ха = DDD (Xou Хи FX Хаи) 


stu 


=>=+12 (ххх, + (Ууу) 


ағы 
=x; +x3, say, 
ха = DED (Xou — -EAF 


= а (Bh хь) 


=ха +3, say. 
To test the Пура ДР Ни: D,=0, we calculate 


ad 1% / ze 


and refer to Fisher's tables of z with degrees of freedom 


n; —1, n;-2n—]1. 
To test the hypothesis H,: C,=0, we calculate 


2 
z=} lo ESP х 2 
n E n—1/ 2n—1 


and refer to Fisher's tables of z with degrees of freedom 
пу=т—1, nj-2n—]. 
To test the hypothesis Н: В. —0, we calculate 


2 2 
23=$ lo “а/ы ) 
iw E: n/ гп—1 


and refer to Fisher's tables of z with degrees of freedom 
n, —n, п —2n —]. 
The original data and the anal 


(77) 


(78) 


(19) 


(80) 


(81) 


(82) 


(83) 


ysis for the group of identical girl 


twins are shown in the tables XXI and XXII ; Table XXI shows also 


the method of.calculating the sums of s 
and for checking the results. 
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quares required in the analysis 


TABLE ХХЛ 


MENTAL AGEs oF IDENTICAL GIRL Twins (COMBINED RESULTS) 


a Mental Ages Diff. Sum Diff. of Sum of 
Pa m inb Wea ee Sums Sums 
air Test 1 Test 2 Xin Ха |I (Xa | 22 Си) 
$ Ха Хаг —Хи | +Х |" Хал) 54 
1 1 163 192 —29 355 7 703 
Е 2 156 192 —36 348 
2 3 113 121 -8 231 12 156 
4 120 102 18 222 
3 5 108 95 13 203 42 448 
6 115 130 -15 245 
4 7 112 126 —14 238 10 486 
8 120 128 -8 | 248 
5 9 134 139 —5 273 14 532 
fT |0 120 139 -19 259 
6 11 136 133 3 269 5 533 
аж | ог 131 133 -2 261 
7 13 131 142 =. 273 10 556 
— | 44 138 145 -1 283 
8 15 124 | 122 [2 246 5 497 
|. № 126 125 1 251 
9 17 216 159 57 375 57 693 
= БЕ: 158 160 -2 318 
10 19 135 | 137 -2 272 7 551 
LL. ] 20 136 143 -7 279 
11 21 136 128 -2 274 30. £ 584 
____| 2 151 159 -8 310 
12 23 121 126 -5 247 n dum 
a | = 113 123 P -10 236 
13 25 102 96 6 198 10 208 
26 112 96 16 5 ME 
n 27 136 | 1H -8 ae d 20 
EROT EE 133 140 Sis a а 
а к= | 691 
5 29 а | їй | 10 S.T 8 
"EE 80 197 162 dà | өм жалын тет” 
| | == "E 
16 31 из | по 4 zu 7 у 
32 121 120 1 2 < 
5 25 49 
17 33 125 134 =й a E 
31 127 107 20 1 


TABLE XXI—continued 


Mental Ages Diff. Sum Diff. of Sum of 
Twin ма Emi Sums Sums 
Pair Test 1 | Test 2 Хи Xu ХХш- | 22(Хаш) 
і Ха Хи —Хиг Ха |“ Хәм) | += 
18 35 138 134 4 272 24 568 
36 144 152 LES 296 
19 37 142 145 —3 287 11 585 
38 158 140 ы 18 298 
20 39 120 124 -4 244 48 485 
40 115 126 —1l 241 
21 41 176 198 -22 374 7 741 
42 175 192 -17 367 
22 43 145 160 —15 305 28 638 
44 143 190 -47 333 
23 45 118 134 —16 252 18 522 
46 123 147 —24 270 
24 4T 225 210 15 435 54 816 
48 189 192 -3 381 
25 49 150 161 -11 311 12 634 
50 163 160 3 323 
26 51 141 148 -7 289 47 531 
52 124 118 6 242 
27 53 164 165 _1 329 13 645 
54 154 162 -8 316 
28 55 145 188 -43 333 
56 179 192 —13 371 s pe 
29 57 126 108 18 |“ 234 7 
58 121 132 -i 253 » E 
30 59 131 153 =22 284 1 58 
60 125 | 149 —24 274 У ы 
31 61 134 151 -17 2 
62 156 | 151 5 305 24 592 
— 
Зит 8,662 8,974 —312 17,636 — 17,636 
Sum of Squares |1,240,328|1,343,858) 18,576 | 5,167,796) 18,600 1 10,316,992 
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TABLE XXII 


ANALYSIS OF VARIANCE OF MENTAL AGES OF IDENTICAL 
Girt Twins (СомвіхЕр RESULTS) 


Variance df. Sum of Squares Mean Square 
Between Tests. . . . . i 785.0323 785.0323 
Between Pairs of Twins . . 30 70,953.6774 2,365.1226 
Within Pairs of Twins. . . 31 4,650.0000 150.0000 
ЕО u ж = е «о = ж 61 8,502.9677 139.3929 
ПЪН) . o w + mis 84,891.6774 = 
T (өз) 
= —— 
12 108e 139.3929 
=0.864 


ћ=т=1; fr=m=61 


ET quo 
#1 ве) 13010020 


=1.417 
fo=m, 230; ја =та=61 
кейін [oe] 
2 -be | 139.3929 
=0.037 


fo=mapl ; Д=т=61 


Interpretation of Results. 


As we see from the test of the 
the two tests is probably sign! 


i ental ages. ^ ; 
> Popes of the hypothesis Н», we conclude that the differ- 
dace s the mental ages of the pairs of twins are significant. 


From the test of the hypothesis Hs: Ва -0, we find that а 
ences between members of the same twin pair m o E САНЕ. 
These differences are of the same order of D we e as шеді erences 
between the estimates of the mental age of the same individual from 


the two tests. 


f the hypothesis Н the difference between 
ficant; the second test gives, on the 
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Comparison with Analysis of Mental Ages of Fraternal Twins (Girls). 


It is interesting to compare these results with the resuits of a similar 
analysis for fraternal twins. Since the above results refer to identical 
girl twins, we have chosen the group of girl (i.e. like-sex) fraternal 
twins for this study. The original data and the results of the analysis 
are given in the following tables; the method used in analyzing the 
results and testing the hypotheses is, of course, the same as that used 
above. 


TABLE XXIII 


MENTAL AGEs оғ Like-SEx (GIRL) FRATERNAL TWINS 
(Сомвімер RESULTS) 


- 3 Mental Ages Diff. SEA Dit. of Sümiof 
win nd. v 
Pair Test 1 | Test 2 Ха Ха 0, pores 
1 Xu Хаг | — Хаг | Хаг ха | 220) 
1 1 134 140 -6 274 49 
2 113 112 1 225 | es 
2 3 159 186 -27 | 345 51 
4 153 141 12 294 689 
5 115 123 -8 238 | 20 | 496 
6 134 124 10 258 
4 7 138 137 1 275 | 25 | 55 
8 141 159 —18 300 
5 9 128 156 -28 284 76 492 
10 118 90 28 208 
6 11 189 192 E 381 88 | 724 
12 177 166 11 343 
Y 13 140 147 —7 287 88 662 
14 208 172 31 375 
8 15 135 137 -2 272 ИВ: | 498 
16 115 111 4 226 
9 17 135 146 -i 281 4 566 
18 134 151 -17 285 
10 19 134 148 -14 282 ЗӨ ЕК, 518 
20 110 126 —16 236 
11 21 112 Ill T 223 19 165 
22 121 121 0 242 
12 23 139 | 154 —15 293 15 571 
24 148 130 18 278 
13 25 146 165 —19 311 63 559 
26 116 132 —16 248 
14 27 120 116 4 236 74 546 
28 136 174 -38 310 
15 29 155 | 146 -П 281 2 560 
30 136 143 -7 279 
16, | 37 109 96 13 205 81 441 
82 121 115 4 6 236 
17 33 115 106 9 221 101 543 
34 167 155 12 322 | 
18 35 141 146 -5 287 23 551 
36 128 136 -8 264 M 
19 37 120 105 15 225 9 441 
38 115 101 14 216 
ЖА : 
Sum 5,130 | 5,216 -86 | 10,346 — | 10346 — 
Sum of Squares | 709,700 | 738,676 | 8,726 2,888,026] 47,046 | 5,729,006 
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TABLE XXIV 


ANALYSIS OF VARIANCE OF MENTAL AGEs oF ІлкЕ-5Ех (GIRLS) 
FRATERNAL Twins (COMBINED RESULTS) 


Variance | d.f. Sum of Squares Mean Square Е 
Between Tests. | 2 97.3158 97.3158 
Between Pairs of Twins | 18 | 23,834.1842 1,324.1213 
Within Pairs of Twins. . . 19 11,761.5000 619.0263 
Error. 37 4,265.6842 115.2888 
Total 75 39,958.6842 
gtd dn с 
97.3158) 
=0.085 


= =37; =} =1 


-— Tuum 
22 256 115.2888 


= 1.221 
т=}=18; т=ј,=87 


619.0263) 
23 =$ log. ,————- 
а те 
=0.840 > 


= f2=19; n= f1 =37 


Interpretation of Results, 


From the test of the hypothesis Hi, we find that the difference 
between the two tests is not significant; i.e. they give, on the average, 
the same mental ages. From the test of the hypothesis Hs, we find 
again that the differences between the mental ages of the pairs of 
twins are significant. From the test of the hypothesis H;, we find 
that the differences between members of the same twin pair are sig- 
nificant; i.e. they are greater than the errors of measurement of mental 
age. It is interesting to note, also, that the errors of measurement of 
mental age are of the same order of magnitude in the two cases. 
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PART 2 


Unequal Numbers of Observations in the Classes! 


In the first part of this chapter, we considered only examples in 
which there were equal numbers of observations in the different classes. 
Since the method may be used in other cases, we shall consider in this 
part the procedure to be followed when the numbers of observations 
in the classes are unequal. The theory underlying the tests, and 
their development, is the same as that considered previously, but the 
method to be followed in calculating the sums of squares required in 
the analysis is different. The difference is not great; it is mainly that 
the arithmetical procedure is somewhat more complicated. 


Example 6. Resemblance of Fraternal Twins in Intelligence. 

In Example 4, we discussed tests of homogeneity and the com- 
bination of results from different experiments, and applied the tests to 
the results for three groups of fraternal twins: 26 unlike-sex pairs, 19 
like-sex girl pairs and 12 like-sex boy pairs. It was found that these 
groups could be considered as samples from a common population of 
fraternal twins and that the results, therefore, could be combined. As 
the groups were of unequal size, the problem of combining the results 
was left for consideration in this section. 

If we denote Бу Хы the mental age of the i-th individual of the ¢-th 
twin pair in the s-th group, we may write 

Xu =A TB, Ca zu (84) 
where s=1, 2, 3; #=1, 2; !=1,2,... . т; п, denotes the number of 
twin pairs in the s-th group. A is a measure of the ability of all fra- 
ternal twins considered, an&is defined as the mean of the mental ages 
for all groups and individuals; B, is a measure of the ability of the s-th 
group; С, is a measure of the ability of the /-th twin pair within the 
s-th group; бш is a measure of the difference between the members of 
the twin pairs and includes also the errors of measurement. Since A 
is defined as the mean mental age for all groups and individuals, it is 
necessary that 

УВ, =0 iw NI 

XC, =0, for each 5 | (85) 

4 


In some problems in education the data аге classified on two criteria and there 
are unequal frequencies in the sub-classes. This type of problem has not been 
considered here; readers interested in these problems should consult: 

Snedecor, George W., and Cox, Gertrude M. Disproportionate Subclass Numbers 
in Tables of Multiple Classification. Research Bulletin No. 180 Ames, Iowa: lowa 
Agricultural Experiment Station, March 1935. Pp. 212. 
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The hypotheses we wish to test are 


H,: B, =0 
Hy: C,,=0 \ (86) 
Define by 
x -XXY[X4-4 =B, — Ca}? (87) 


and minimize x? 
(1) with regard to all the quantities, to obtain the absolute 
minimum value of x”, х2; 
(2) with regard to the quantities remaining if H, is true, to 
obtain the first relative minimum value of x^, ха; 
(3) with regard to the quantities remaining if H is true, to 
obtain the second relative minimum value of x’, х5. 


We obtain 
1 = 
A= y EXXXa-X.. 


where № =2))n, 
i (88) 


1 = - == 
В,= Ex МХК =, —ÀÁ Я 
21, 


ЖЕ. 


C, 23X Xu Хо. = Хи —X,.. 
оо р 

SIBE (Хи – Хо) ] (89) 
ха = ZED (Хи -Ra Хи X. 


: (Ух. (xxxx,? 
= ру 13 Lo sti 
5 т; N 
=xetxi, say. (90) 
xà -ХХХ(Хы =) 
: (ххх, 
=%+%% (Ха) 
5 n, 
=x5+x3, say. (91) 
To test the hypothesis Hy, we calculate 
2 7.2 
ДУ (92) 
AL fs 
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and refer to Fisher’s tables of 2 with degrees of freedom 


n =f, =2, п 7 fa Xin, =57. 
To test the hypotheses И, we calculate 


2 2 
2; =} log, ЕШ а) (93) 


and refer to Fisher's tables of 2 with degrees of freedom 


n = fa X(n, —1) =54, and ж = f = Xin, =57. 
5 5 


We see from equations (89) and (91) that we can obtain the values 
of x? and x3 from the analysis given in Example 4; the sum of the 
three "within pairs" sums of squares will give us the value of x2, and 
the sum of the three ‘‘between pairs” sums of squares will give us the 
value of ха. Тһе value of xi, however, must be calculated from the 
totals given in Tables X, XII, XIV. In this calculation, we have 
to divide each square, (XXX), by п, before summing for s: this is the 

== 


i 
difference in procedure mentioned above. We find 


| , a. | (6182)? , (8292): а 
| mimm 


(15656)? _ 67.8502 


114 


The complete analysis in the usual form is given in the following table. 


TABLE XXV 


ANALYSIS OF VARIANCE OF MENTAL AGES OF FRATERNAL TWINS 
(CoMBINED RESULTS) 


Variance df. Sum of Squares .| Mean Square 
Between groups. 2 67.8502 33.9251 
Between pairs . . + - 54 40,377.4831 747.7312 
Within pairs . - - + - 57. 14,296.0000 250.8070 
а > s + = з 2 113 54,741.3333 ———— | 


To test Hı, we calculate 


m 
Е: 50.8070 |, _ 1 99 
ae ote 33.9251 
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and refer to Fisher’s tables of z with degrees of freedom 7, =57 and 
п» —2 (see Note 2, chapter 1, page 16). We find z; is less than the 5% 
point so we accept H, and conclude that the differences between the 
average ability of the groups are not significant. It is not necessary 
to calculate z, in this case, however, as it is clear from the values of the 
mean squares that we should accept H, (see also Example 2(5)). 

To test the hypothesis И», we calculate 


970 аза 
oe 9*1 250.8070 


and refer to Fisher's tables of 2 with degrees of freedom 71--54 and 
пу=57. Since 2;=0.546 is greater than the corresponding 1% point, 
we reject the hypothesis Н». We conclude that the differences be- 
tween twin pairs are significant or, as in Example 1, that the intraclass 
correlation is greater than zero. 


\ =0.546 


Example 7. Comparison of the Ability of Pupils in Different Classes 
in the Same Grade. 


Since there are generally more pupils enrolled in each grade ina 
High School than can be accommodated in a single classroom, it is 
necessary to divide the pupils into classes. If we know that the 
classes are equal in ability and variability, then we may make a direct 
comparison of their progress, but if this is not known we must, before 
making the comparison, determine the relative ability and variability 
of each class. The present example, therefore, is concerned with the 
problem of determining whether the ability and variability of different 
classes are equal. 


The data used in the two illustrative cases considered here refer to 


the scores made on an intelligence test which was given to the pupils 
in two grades, IX and X, in an Ontario High School; there were six 


classes in Grade IX and fivein Grade X. Since 343 pupils were tested 
altogether, the tables giving all the original scores would be too bulky 
to be included here, but the relevant sums and sums of squares for 
each class and grade are given in the following tables. 
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TABLE XXVI 
Sums AND SUMS or SQUARES OF SCORES FoR EacH Crass (Grape IX) 


Number Sum of Sum of Squares Sum gf ae 
Class 4 Scores of Scores эй. 
Pupils vy тү? ху? _ (Аа 
IXau ЕХ Хи 1 
[ns] 1 n t 
"s 
A 32 530 10,770 1,991.8750 
B 34 704 17,550 2,973.0588 
е 31 446 7,210 793.3548 
D 34 792 21,556 3,107.0588 
E 25 520 12,336 1,520.0000 
Е 30 510 9,518 848.0000 
Тога! 186 3,502 78,940 11,233,3474 


TABLE XXVII 
SUMS AND SUMS OF SQUARES OF SCORES FOR Елсн CLASS (GRADE X) 


Class Pupils Xa zx sxi- Сх.) 
[ns] t ! Џ m 
A 33 928 28,030 1,933.5152 
B 28 760 22,750 2,121.4286 
С 31 1,013 35,287 2,184.7742 
p^ D 34 1,335 56,637 4,218.6176 
ани 31 748 2 21,336 | 3,287.4839 
Total 157 4,784 164,040 13,745.8195 


If we denote by Ху the score made by the i-th pupil in the s-th 
class, the basic assumption in the analysis is that we may write 

Xa =A +B, 25 (94) 

where s=1, 2,...., Ё; !=12,.... 5 Msi Ms denotes the number of 

pupils in the s-th class and k denotes the number of classes. A is a 

measure of the ability of all the pupils in the grade, and is defined as 


the mean score for all individuals and classes; B, is a measure of the 
ability of the s-th class; 2s, isa measure of the differences within classes; 
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> 


it is assumed to be normally distributed about zero with constant 
standard deviation c. It follows from the definition of A that 


2 B,=0 (95) 


The hypothesis we wish to test, namely, that the classes are of equal 
ability, may be written 


Н,:В,-0 (96) 


If we assume that the standard deviation с is constant, we are 
assuming that the variability of the scores is the same for each class. 
This assumption may not be satisfied in practice, and, in fact, difference 
in variability is one of the factors in which we are interested, so we 
must first test the hypothesis 


Но: в, =с (97) 
where c, denotes the standard deviation of the scores in the s-th class. 
If this hypothesis is accepted, we conclude that there is no difference 
in variability and proceed to the test of the hypothesis H,. If, оп the 
other hand, we reject the hypothesis Hy, we conclude that the classes 
differ in variability and, in addition, we find that we cannot make an 
exact test of the hypothesis Hy. This is one limitation of the analysis 
of variance method, but in this particular case it does not matter much, 
from the practical point of view, as we cannot 
progress of classes of different variability. 


The test of the hypothesis H, may be made by using the method 
discussed in Examples 4 and 5. 


The computation of Тл, for the six 
classes in Grade IX, may be carried out as shown in the following table. 


directly compare the 


TABLE XXVIIÍ 
CALCULATION оғ L; FOR THE TEST OF THE НУРОТНЕ515 


Ho: 0,=0 
(Grave FX) 

п; log ns ns log ns 9; log 0; ns log 0; 

32 1.5052 — 1,991.8750 3.2993 m 
= 34 1,5315 = 2,973.0588 3.4732 — 

31 1.4914 E 793.3548 2.8995 = 

34 1.5315 === 3,107.0588 3.4923 =: 

25 1.3979 --- 1,520.0000 3.1818 = 
а 90; iL 14771 — 848.0000 2.9284 = 

186 En, log п; =277.7997 11,233.3474 2а log 0;--599.6866 
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We obtain Lı =0.890, and referring to Nayer's tables with k =6 and 
mean л, of 31 we find that 2. is less than the 1% point. We reject the 
hypothesis Но and conclude that the classes are of different variability; 
we cannot make a further analysis, therefore, to test the hypothesis H, 
in this case. 

Turning to the results for Grade X, the computation of L4, to test 
the hypothesis Но for the five classes in this grade, may be carried out 
as shown in the following table. 


TABLE XXIX 
CALCULATION OF Li FOR THE TEST OF THE НҮРОТНЕ5І5 Ho:0,=0 
(СКАРЕ X) 

т log па ns log п; 9; log 0; ns log 0; 
33 1.5185 --- 1,933.5152 3.2863 — 
28 1.4472 = 2,121.4286 3.3266 —— 
31 1.4914 — 2,184.7742 3.3394 — 
34 1.5315 — 4,218.6176 3.6252 --- 
‚ 31 1.4914 --- 3,287.4839 3.5169 —— 
157 Zn; log 1; 235.1661 13,745.8195 ans log Ө; =537.3952 

$ 


We obtain 2, =0.961, and referring to Nayer's tables with k=5 and 
mean 7, of 31.4 we find that Ly, is greater than the 5% point. We accept 
Но, and conclude that the classes are of equal variability. 
Since our assumption of constant standard deviation, 
satisfied, we may proceed to the test of the hypothesis Н. The de- 
velopment of the test is similar to that considered in Example 1 and 
will not be repeated here; the Only difference, in fact, is that in this 
case we have unequal numbers of observations in the different classes. 


0,-—0, 15 


The complete analysis is given in Table XXX. 
TABLE XXX 
ANALYSIS OF VARIANCE OF SCORES IN DIFFERENT CLASSES IN GRADE X 
Variance d.f. Sum of Squares Mean Square 
Between Classes . 4 4,519.3015 1,129.8254 
Within Classes 152 13,745.8195 90.4330 


Total 156 18,265.1210 - 


The within classes sum of squares may be obtained directly from 
' the last row of Table XXIX, У, = 13,745.8195 ; the between means of 


classes, or between classes for short, sum of squares is calculated from 
the totals given in the third column of Table XXVII as follows (see 
explanation in Example 6) 
(1335)? E: (1013)* + (928)? 760)? _ (4784)? 
31 33 28 157 


—4,519.3015. 


(748)? | ( 
34 5 31 | 


The total sum of squares (to be used in checking the calculations) is 
2 
164040 — 4784)" _ 18 965.1910 
157 


To test the hypothesis Ни, we calculate (from Table XXX) 


(1,129.8254 


= log, | 11-49-5494 
2) 5004950 


\ -1.283 


and refer to Fisher's tables of z with degrees of freedom 71-4 and 
N2=152. Since z is greater than the 1% point, we reject the hypo- 
thesis Н, and conclude that the classes differ in ability. 

In the first case (Grade IX), we found that the classes differed in 
variability, and probably in ability also but we could not make an 
exact test of the significance of these latter differences. 
case (Grade X), we found that the classes were of equal variability 
but differed significantly in ability. A comparison of the progress of 
these classes, therefore, would be valid only if these factors were taken 
into consideration. 


c 


In the second 


Example 8. Test of the Linearity of Regression. 
The statistical method of regression, 


widely used in education and other fields, 
arises of determining whether 


considered adequately represents the obse; 


linear and non-linear, is 
In every case the difficulty 
articular type of regression 
rved data. The analysis of 


d non-linear regression. As 


; у n straight regression line, we 
shall discuss in this example only the application of the method to the 


problem of testing the linearity of Tegression. The method will be used 


in analyzing the results from two experiments: in the first the regres- 
sion is linear but in the second it is not. 


It should be pointed out that underlying the statistical method of 
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correlation, which is so widely used in education, are the assumptions 
that, (1) the regressions are linear in form, i.e. that the relationship 
between the variables can be adequately represented by a straight line, 
and (2) the variances of the different arrays are equal. As the validity 
of these assumptions should be tested for every case, the methods dis- 
cussed below should be of particular value in educational statistics. 

In the case considered here, the practical problem was that of 
determining whether the relationship between the scores of the same 
individuals on two mental tests was linear in form. If we denote by 
X and y the scores on the first and second tests, respectively, then, if 
the regression is linear, the regression function may be written 


Y=a+b(X—-X) (98) 


where a and b are two parameters to be estimated from the data; b is 
the regression coefficient of y on X, Y is the expected value of y for 
each X, and X is the mean value of the distribution of X. The same 
methods may be used, of course, in studying the form of the regression 
of X on y. 

For each selected value of X we shall generally have several values 
of y; these form what is termed an array. If the data are grouped, 
as in our case, the mid-points of the class-intervals may be taken as 
the selected valuesof X. If we denote by y, the score (on the second 
test) of the ż-th individual in the s-th array, we may write 


Y,,=A+B, +25 (99) 


where 1=1,2,.... па $5715 2... k; k is the number of arrays and 
n, is the number of individuals (i.e. observations) in the s-th array. 
A is а measure of the general ability of the individuals in the group 
tested and is defined as the mean of the scores, on the second test, for 
all individuals and arrays; B, is a measure of the ability of the indi- 
viduals in the s-th array, i-e. as determined by the scores on the second 
test; 2, is a measure of the errors of measurement by means of the 
second test, including errors of grouping, etc., which are independent 
of the value of X. Itis assumed to be a random variable normally 
distributed about zero with the standatd deviation е, say, constant for 


all arrays. 


The analysis here, therefore, is similar to that considered in 


Example 7; the total sum of squares may be broken up into two parts, 
ance between means of arrays and the other the 
The relevant data are given, in summary 
Before proceeding with our analysis, 


one giving us the vari 
variance within arrays. 
form, in the following tables. 
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TABLE XXXI 
Case I. Sum AND Sum oF SQUARES OF SCORES ІМ EACH ARRAY 


Sum of 
Sum of | Squares (Ху)? (Хум)? 
of ы 2 t 
Array | Value т, Scores CoD L О%- 
Мо. of пз * 2 
х. Zw | 2% 
1 8.5 19 249 3,533 3,263.2105 269.7895 
2 10.5 20 311 5,729 4,836.0500 892.9500 
3 12.5 32 613 12,429 11,742.7813 686.2187 
4 14.5 35 727 16,293 15,100.8286 1,192.1714 
5 16.5 34 711 16,299 14,868.2647 1,430.7353 
6 18.5 40 956 24,444 22,848.4000 1,595.6000 
7 20.5 52 1,314 34,748 33,203.7692 1,544.2308 
8 22.5 38 1,066 31,472 29,904.1053 1,567.8947 
9 24.5 29 885 27,891 27,007.7586 883.2414 
10 26.5 35 1,085 34,551 33,635.0000 916.0000 
11 28.5 39 1,267 42,405 41,161.2564 1,243.7436 
12 30.5 29 1,062 39,540 38,891.1724 648.8276 
13 32.5 22 850 33,734 32,840.9091 893.0909 
14 34.5 30 1171 46,597 45,708.0333 888.9667 
15 36.5 19 809 35,385 34,446.3684 938.6316 
16 38.5 17 714 30,532 29,988.0000 544.0000 
17 40.5 21 979 46,371 45,640.0476 730.9524 
18 42.5 19 950 47,982 47,500.0000 482.0000 
19 44.5 14 707 36,315 35,703.5000 611.5000 
20 46.5 16 842 44,740 (| 44,310.2500 429.7500 
21 48.5 22 1,215 67,903 67,101.1364 801.8636 
22 50.5 18 1,064 63/280 62,894.2222 585.7778 
23 52.5 14 912 59,940 59,410.2857 529.7143 
Total 614 20,459 | 802,313 782,005.3497 | 20,307.6503 


however, we must test the validity of our assumption that the stari- 
dard deviation c is constant for all arrays. 


We may test the hypothesis that с is constant for all arrays by the 
method used in the previous example. .For the first case, we find 
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TABLE XXXII 


Case 2. SUM AND SUM OF SQUARES OF SCORES IN EACH ARRAY 


Sum of 
Sum of | Squares Уу)? (Ey 
Array | Value Ns Scores of E зы) Zy- ya) 
No. of Scores т, mé Ns 
X; Ози Буг 
1 6.5 18 205 2,723 2,334.7222 388.2778 
2 8.5 16 194 2,720 2,352.2500 367.7500 
3 10.5 17 231 3,365 3,138.8824 226.1176 
4 12.5 22 313 4,905 4,453.1364 451.8636 
5 14.5 27 394 6,256 5,749.4815 506.5185 
6 16.5 22 386 7,136 6,772.5455 363.4545 
7 18.5 32 563 10,501 9,905.2813 595.7187 
8 20.5 24 508 11,171 10,542.0417 628.9583 
9 22.5 34 681 14,395 13,640.0294 754.9706 
10 24.5 34 782 18,704 17,986.0000 718.0000 
11 26.5 46 1,028 24,548 22,973.5652 1,574.4318 
12 28.5 85 901 24,023 23,194.3143 828.6857 
13 30.5 33 900 | 26,022 24,545 4545 1,476.5455 
14 32.5 30 872 26,452 25,346 1333 1,105.8667 
15 34.5 25 760 | 23.598 23,104.0000 494.0000 
16 365 31 1,023 34,555 33,759.0000 796.0000 
17 38.5 17 607 22,663 21,673.4706 989.5294 
18 40.5 21 766 | 28,352 27,940.7619 411.2381 
19 425 22 2845 33,405 32,455.6818 949.3182 
20 44.5 15 635 27,509 26,881.6667 627.3333 
21 46.5 20 900» | 41,672 40,500.0000 1,172.0000 
22 48,5 21 960 44,348 43,885.7143 462.2857 
23 50.5 16 781 38,585 38,122.5625 462.4375 
24 52.5 19 1,006 54,262 53,265.0526 996.9474 
25 54.5 17 1,061 66,961 66,218.8824 742.1176 
Total 614 17,297 | 598,831 580,740.6305 | 18,090.3695 
у п,=27, is 


Тл =0.970 which, for k=23 and mean array frequenc 
во we may assume that c is constant. The 


greater than the 5% point, Е 
his case is given іп the following table: 


first analysis of the scores for t 
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TABLE XXXIII 
ANALYSIS OF VARIANCE OF SCORES ON SECOND ТЕЅТ (САВЕ 1) 


Variance | d.f. Sum of Squares Mean Square 

Between Means of Arrays . 22 100,294.1429 4,558.8247 
Within Arrays = v æ ə 591 20,307.6503 34.3615 
оаа RM шт 613 120,601.7932 ———— 


The sums of squares are obtained from the totals given in Table XXXI. 
The within arrays sum of squares is simply the total of the last column; 


the between means of arrays sum of squares is calculated from the 
totals of columns (4) and (6), i.e. 


782,005.3497 — (озу —100,294.1429 


and the total sum of squares from the totals of columns (4) and (5), i.e. 


802,313 — (20459) _ 199 601.7932 
614 


The test of the hypothesis of a constant c is similar for the second 
case; we find L, 20.926, k=25 anda, 721.6, which is approximately 
equal to the value given for the 5% point, so again we may assume that 
cisconstant. The first analysis of variance of the scores for this case 
is given below; the sums of squares are calculated as explained above. 


TABLE XXXIV 
ANALYSIS OF VARIANCE OF SCORES ON SEcoNp Test (Case 2) 


Variance d.f. Sum of Squares Mean Square 
Between Means of Arrays . 24 93,466.6745 3,894.4447 
Within Arrays . . . , 589 18,090.3695 30.7137 
pic e а ас 613 111,557.0440 ---- 


The hypothesis we wish to test is that the regression of y on x is 
linear, і.е. 


Ни: Ү,-а--ҚХ,-Х) (100) 
where У; is the expected value of У for the s-th value of X. If this 
hypothesis is true, we may write a+6(X,—X) instead of A+B 
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,in 


equation (99), and the test of H, may be developed as in the previous 
examples. We write 


x -XX64-4-—By (101) 
54 
and minimize x? with regard to all the parameters to obtain the abso- 
lute minimum value of x”, x2. 


We have Ке 


х => уп -A | (102) 
272 а 


т, 


which is the within arrays sum of squares. The second step is to 
minimize x? with regard to the parameters remaining if H; is true, i.e. 
minimize 


x-XX іш-а-Қх,-Х))? (108) 


with regard to the parameters а and 5 to obtain the relative minimum 
value of x”, х2. We obtain 


ee i 5 

= yu, 2209 9. (104) 
D[(X,-X) ы)! 

c (105) 
Xx». -xy] 

4 (x) (уыл (510, –—20 Фу )] + 

NT 1 Li COME 1 me t 
Xr xX 4 | 5n DIPIERES (106) 
(107) 


2 
-хі-Ехі, say. 
Xi is seen to be equal to the sum of squares ascribable to between 
means of arrays minus the quantity 


[Ef (x, = о 


; (108) 
Xxina.-xy] 


j= 


To test the hypothesis Hi, we calculate 


Ыр tm 


and refer to Fisher's tables of 2 with degrees of freedom т =k —2 and 
по = Уп, Е. It is convenient, and also customary, to present the 


wi 
о 

03 

га 


ES 


5 H H 
analysis in the form shown in the following tables; / is entered as the 
"variance due to linear regression” and xi as the “variance due to 
departures from linear regression". 
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d TABLE XXXV 


ANALysis өс VARIANCE OF SCORES ON SECOND Test (САЅЕ 1)— 
. COMPLETE ANALYSIS 


"Variance d.f. Sum of Squares Mean Square 
Due to Шпеат Regression . 1 99,203.3378 99,203.3378 
Due to Departures from 
ti ас Regression: a ле 21 1,090.8051 51.9431 
Within Arrays . . · - 591 20,307.6503 34.3615 
ТОН, 4*8 €» ox 613 120,601.7932 ——— 


For the test of the hypothesis Ни, in Case 1, we find 


=} log, [219481 


Blog, 121.991} — 0.207 
DUE ert 


and we see from Fisher's tables, entered with degrees of freedom n, =21 
and пз =591, that this value is less than the 5% point of the distri- 


bution of z. We accept the hypothesis H, and conclude that the 
regression is linear. 


TABLE XXXVI 


ANALYSIS OF VARIANCE OF SCORES ON SECOND ТЕЗТ (CASE 2)— 
COMPLETE ANALYSIS. 


Variance d.f. Sum of Squares Mean Square 
Due to Linear Regression il 89,034.3367 89,034.3367 
Due to Departures from Ш 
Linear Regression . . 23 4,432.3378 192.7103 
Within Arrays . . . . 589 z, 18,090.3695 30.7137 


Tota 550202 + 613 111,557.0440 = 


For the test of the hypothesis Hy, in Case 2, we obtain 


сө тат 

2 log, {1927103 0.918 

We find from Fisher’s tables of 2, entered with degrees of freedom 
п! =23 and п: —589, that this value is greater than the 1% point. We 
reject the hypothesis Ну, therefore, and conclude that the regression 
is non-linear in form. It will be noticed, however, that the greater 
part of the variance is accounted for by the linear regression factor. 
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CHAPTER III Қы. © 


ANALYSIS OF COVARIANCE 


It was pointed out in the first chapter that the analysis of covari- 
ance is an extension of the analysis of variance method. If we are 
considering two variates, for example, we may make a separate 
analysis of the variance of each and if the two variables are related 
we may make, at the same time, an analysis of the covariance. In the 
analysis of covariance we break up the sum of products into parts 
ascribable to different factors in much the same way as we break up 
the sum of squares in the analysis of variance. In the examples which 
follow, we shall consider only the case of two variates, but the method 
may easily be extended to cover the case of three or more. 


Example 9. Resemblance of Fraternal Twins (Unlike-Sex Pairs) in 


Intelligence. 

In Example 4, we discussed the analysis of the mental age scores 
of fraternal twins, and in Example 1 we showed that a similar analysis 
could be made for chronological age. In this example we shall con- 
sider the combined analysis of mental age and chronological age in the 
case of the fraternal twins,(unlike-sex pairs). The analysis of the 
mental age scores for this group may be found in Example 4, Table 
XVI. 
Denote by X; the mental agè of the -th member of the /-th twin 
pair, and by У; the chronological age of the i-th member of the ¢-th 
twin pair. Following the method of Example 1, we write 


Xa =A C254 (110) 


and o 
Y,=B+D, (111) 
‚ п ; where п is the number of twin pairs. 


where i=1, 2; і-1,2,.... 
the members of the same 


In the case of chronological age, of course, 
twin pair are of the same chronological age, i.e. Yu = У = Y, so the 
measure of the differences within pairs of twins is zero. The total 
variation in chronological age, therefore, is ascribable to differences 


between pairs of twins. 
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The differences between the mental ages of the pairs of fraternal 
twins may be due partly or wholly to differences in chronological age. 
Our problem, therefore, is to determine what proportion of these differ- 
. ences may properly be ascribable to differences in chronological age, 
and adjust our analysis accordingly. If we may assume that the rela- 
tionship between mental age and chronological age is linear in form, 
then, since Y, denotes the chronological age of the /-th twin pair, we 
may write 


X4 ab Y, S, (112) 


where a and b are constants to be estimated from the data; b is the 
regression coefficient of mental age on chronological age, and Sj, is a 
measure of the differences between the mental ages of members of the 
same twin pair and differences between the mental ages of pairs of 
twins not ascribable to the factor of chronological age. 


Following the usual method, we minimize 
x: -XX[X,—-a-bY,]: (113) 
ЗЕ 


with regard to a and b to obtain the relative minimum value of x”, х2, 
say. We have 


= 1 = 
а= = DEXa УХ ү] (114) 


(ШУ) (TUX) 

> У (БУХ) cr d 

(EYy (115) 
и 


хи) 
x = (Хи- Хы ЕСЕ = саз 


EY) EEX) F 
È ¥,(DXy)} — eei 


2 È Y? z ЕД 


п 


=x: txi, say. (116) 
68 ] 


The portion of the variance ascribable to chronological age, there- 


fore, is 
(УХ 
[шык = == | 
1 | ғ Д (117) 
2| YY?2 ——— 
" п 


since the other two quantities in (116) are simply the within апа be- 
tween pairs sums of squares for mental age. То obtain xj we subtract 
l from the between pairs sums of squares. 

The necessary calculations may easily be made if the data are 
arranged as shown in the following table. 
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TABLE XXXVII 
MENTAL AND CHRONOLOGICAL AGES оғ UNLIKE-SEX FRATERNAL TWINS 


Chronological 
Twi Mental Age Difference Sum Age 
Pair (Mos.) |Хи— Хо Хи+Хи (Mos.) 
Хи t 
97 
1 13. 207 116 
110 
103 
2 26 232 121 
129 
қ 139 
3 8 270 135 
131 
151 
4 19 283 176 
132 
F 180 
5 40 320 158 
140 
118 
6 6 242 111 
124 
142 
T 9 275 | 142 
Е 133 | 
158 
Б 2 318 154 
160 
" 145 К 
17 27. 5 
128 3 135 
10 145 
5 Н 285 
140 8% T 
i 126 
9 
17 ы 128 
f 130 | | 
К 136 8 26 | ы 
13 139 
66 | у 
205 р 344 140 
ii 130 
19 
11 | 241 108 
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TABLE XXXVII—Continued 


à Chronological 
Twin Mental Age Difference Sum ge 
Pair (Mos.) | Хи—Х» | Хи+Ха (Mos.) 
Хи б 
138 
18 25 251 126 
113 
162 3 
16 22 302 145 
140 
146 
17 31 323 160 
177 
144 
18 3 285 132 
141 
135 
19 1 269 154 
134 
120 
20 7. 247 135 
127 
116 
21 0 232 144 
116 
177 
22 39 315 164 
138 
125 
23 о з 247 128 
122 
122 
24 * 36 280 146 
158 
149 
25 15 313 152 
164 
1 3 
26 К 3 319 136 
158 
Total 7,182 — 7,182 3,673 
Sum of Squares 1,013,998 | 13,328 2,014,668 526,963 
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The results are generally presented in the form shown below. 


TABLE XXXVIII 
ANALYSIS OF VARIANCE AND COVARIANCE OF MENTAL AND 
CHRONOLOGICAL AGE SCORES 


Sum of Squares |Sum of Products| Sum of Squares 


Variance d.f. оон (С.А.) (М.А.) | (Mental Age) 
ge 
Between Pairs of Twins| 25 16,162.231 9,693.385 15,389.308 
Within Pairs of Twins| 26 0 0 6,664.000 
Toal 2.2. 210 16,162.231 9,693.385 22,053.308 


The quantities entered in the above table are calculated as follows: 


Mental Age 


Between Pairs: 2014608 EIL 
2 52 


—15,389.308 


Within Paire: == —6,664.000 


2 
Total: 1013998 — wey =22,053.308 


Chronological Аре 
Between Pairs: 2(526963) — еса. =16,162.231 


Within Pairs: 0 
Total: 1053926 — 59) —16,162.231 
Products (C.A.)(M.A.) 
Between Pairs: (a16)(207) + (121)(232) 4-. ..... 


A +(136)(319)} — 01828679) = 9,003.385 
Within Pairs: 0 


Total: { (116)(97)-+(116)(110)-+. . .4-(136)(161) 
+(136)(158)} — (1800710) = 9,693.385 
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It will be seen that the between pairs sum of products gives us the 
quantity inside the brackets in the numerator of (117), and the denom- 
inator in the same equation may be obtained from the corresponding 
entry in the chronological age column. We find 


(9693.385)? 
16162.231 
The sums of squares for the analysis of mental age corrected for the 


effect of chronological age; or the adjusted sums of squares and mean 
squares as they are termed, are shown in the following table. 


=5,813.660 


TABLE XXXIX 


ANALYSIS OF VARIANCE OF MENTAL AGE SCORES OF FRATERNAL TWINS— 
ORIGINAL AND ADJUSTED SUMS OF SQUARES AND MEAN SQUARES 


Original Analysis Adjusted Analysis 
Variance 
ағ Sum of Mean df. Sum of Mean 
P Squares Square in Squares Square 
Between Pairs . . | 25 |15,389.308 | 615.572 24 | 9,575.648 | 398.985 
Due to effect of 
Chronological Age . — — = 1 | 5,818,660 | 5,813.660 
Within Pairs . . . | 26 | 6,664.000 | 256.308 26 | 6,664.000 | 256.308 
ОТА; $e s 51 |22,053.308 = 51 |22,053.308 — 


The adjusted between pairs sum of squares and mean square give 
us, as we see from (116), a measure of the differences between twin 
pairs in mental age freed from the influence of chronological age. To 
test the hypothesis that these adjusted differences are zero, we cal- 


culate 
2 2 
АЕ И (118) 
т/ ma 
and refer to Fisher's tables of z with degrees of freedom ny =n —2 and 
пут. In our example we have 


мела, 
log, 299-959. 0201 
z=} log. Ее 


and from Fisher's tables, entered with degrees of freedom пл =24 and 
па =26, we find that 2 is less than the 5% point. We accept the hypo- 
thesis, therefore, and conclude that when the factor of chronological 
age is removed, the differences in mental age between pairs and within 
pairs of fraternal twins are of the same order of magnitude. 
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We may obtain two measures of the degree of relationship between 
mental age and chronological age from the results shown in Table 
XXXVIII. From the first row, for between pairs, we һауе 


9693.385 


=—— = 0.615 
"= V (16162.231) (15389 308) 
and from the last row, for the total we have 
9693.385 
7% 0.513 


7 +/(16162.231) (22053.308) 


The first, 7 =0.615, is the better measure of the degree of relationship; 
in the second the relationship is masked by the inclusion of the within 
pairs differences in mental age. 


Example 10. Relationship Between the Scores of Pupils on Two 
Mental Tests. 

In measuring the relationship between the scores of pupils on two 
mental tests, we frequently meet with the problem of deciding whether 
or not it is permissible to combine the results from several classes or 
grades. There is an additional problem here which is not generally 
recognized: that of determining the effect of combining the results 
on our measure of the relationship between the variables. As these 
problems are related, we shall consider both of them in this example 
and apply the methods to the results for five classes in Grade X. Part 
of the data has already been analyzed in Example 7; we found that 
the classes were of equal variability but differed significantly in ability. 
In the following analysis we assume that the classes are of equal vari- 
ability; this assumption is satisfied in our case but, of course, if it is 
not satisfied we cannot apply the methods developed here. 

If we denote by X,, and Y,, the scores of the ¢-th individual in the 
5-th class on the first and second test, respectively, then we may write 


X474 B, +2, (119) 
Ya = СЕР, Fu, (120) 
where. s=1, 2,22... В; ра], 2%. 


+» s; k denotes the number of 
classes and z, the number of individuals in the s-th class. For the 


first and second test, respectively, A and С are measures of the ability 


of all the pupils in the grade and are defined as the mean scores for all 


individuals and classes; B, and D, are measures of the ability of the 
s-th class; 2, and u, 


i Za are measures of the differences between indi- 
viduals within classes, and are assumed to be normally distributed 
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about zero with constant standard deviations, c; and оз, say. It fol- 
lows from the definition of A and C that 


DB, =0 
íi (121) 


22.-0 


We will consider the relationship of the scores оп the second test 
to those on the first, i.e. the regression of У, on Хи; the same method 
may be used, of course, in investigating the relationship of the scores 
on the first test to those on the second. The relationship is actually 
more complex than it appears as we must distinguish between two 
relationships: (1) the relationship between the estimates of the ability 
of the classes, and (2) the relationship between the estimates of the 6 
abilities of the individuals within each of the classes. If the relation- 
ship within classes is the same for each class, then we have only the 
relationships between and within classes, but we must first determine 
whether there is a common within classes relationship. 

Assuming that the relationships are linear in form, we may write 


Ул=а +0B, Нели + Sst (122) 
‘or, using the estimates of В, and Zs from our previous analysis, 
Yy =0 +(X; —X.) ka Ga 7 X2) Sa (123) 


where a, b and с, are constants to be determined from the data; b and 
c, are the regression coefficients for between and within classes, respec- 
tively, and S,, is a measure of the errors of estimating the scores on the 
second test from those on the first. We assume, further, that the 
quantity 5, is normally distributed about zero with constant stan- 
dard deviation e, constant for all classes. One of the first tests we 
shall have to make is a test of the validity of this assumption. If it is 
not satisfied, then we shall not have a common standard deviation, c, 
but a standard deviation, о, say, for each class. It is necessary, there- 
fore, to test the hypothesis 

Hy;:o,—60 (124) 


before we proceed with the analysis. 
For each s, equation (123) may be written in the form 


Y, -d-Fe(Xs —Х,.)+ Зи (125) 
where d is a constant to be determined from the data. Denote by 
(126) 


=D { Yu—d—o,(Xu-Xe)}? 
1 
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and minimize x? with regard to d and с, to obtain the minimum value 
of x2, which we may denote by 6,, as our estimate of f,o?. We find 


(BX ФР УДЫ 
Ya Ё E Х.У.) а) 
0,=DY3— E ale : = E (127) 
1 п; xxi- b st) 
L п; 


To test the hypothesis H;, we proceed as in Examples 4 and 7, follow- 
ing the method developed by Welch (13). The original data, in sum- 
mary form, are given in the following table. 


TABLE XL 
DATA RELATING TO THE SCORES or PUPILS ON THE Two MENTAL TESTS 
Sum of 
Number| Sum of Sum of Sum of Sum of Products of 
Class of Scores on | Scores on | Squares of Squares of Scores on 
Pupils | First Test | Second Scores on Scores on First and 
Test First Test | Second Test | Second Test 
5 Ns zx и ХҮ, zx zy У РиХи 
1 t t 
1 33 928 3,753 28,030 439,065 108,513 
2 28 760 3,122 22,750 357,918 87,923 
3 31 1,013 3,782 35,287 478,686 128,069 
4 34 1,335 4,786 56,637 694,860 195,525 
5 31 748 3,366 21,336 383,646 86,685 
Total 157 4,784 18,809 164,040 2,354,175 606,715 


From these data we calculate the 
value of Гл for the test of the hypothesi 
with k=5 and degrees of freedom f= 


other quantities required in tl 
XLI. 


quantity 6, for each class, the 
s Hı, and refer to Nayer's tables 
29.4. The values of 6,, and the 
зе calculation of L, are shown in Table 
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TABLE XLI 
VALUES OF 0;, AND CALCULATION OF L, 


Class ns log п; ns log ns 9; log 6; | ns log 0: 

1 33 | 151851 — | 7,671.8638 | 388490 == 

2 эв | 144716 — || 5,039.2140 | 3.70236 жалын 

3 31 1.49136 --- 8,083.2055 3.90758 —— 

4 34 1.53148 — 7,453.5886 | Г 3.87237 — 
> 31 | 149136 те” ота» | 395776 | — 
Total | 157. Zn, log ns=235.16609 |37,321.0148 | Уи; log 0: =607.35390 


We obtain L;=0.988, and from Nayer's tables we find that Li is 
greater than the 5% point, so we accept the hypothesis Hı. The as- 
sumption of a constant standard deviation is valid, therefore, and we 


may proceed with the analysis. 
Denote by 
қ ж = УУ (У, а Хо =R) a Ka KA (128) 
st 


and minimize ха with regard to all the parameters a, b and с, to obtain 


2 
the absolute minimum value of X5, x2: 


We have 
а= LXXxvY.oN-Em (129) 
Nst 5 
ях] (ХР) УХ) 
t t m. xw Ж.Ж 
~ т | м (130) 
Е |] Ух 
X n Жа Ж: 
5 п; N 
(ZX,) Yu) 
zi У Ха -~ Ре £ x 
5 1 
си COR ( 
SX 
t п; 
іог5-1,2,...., Ё. 
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x EN Yi— = 
ы (®БЖҮШ(®®Х.„) | 
Al : п, Е N 
ГЕ“ рај 
2 п; ~ N 
CIEST 
ЕР шілес теа 
= x x (132) 
j а С. 
t п; 


If there is а common within classes relationship, then c, ^ c and we 
may write 


m =a +b(X,. -Х.)--(Х, —X,) +S, (138) 


The hypothesis we shall have to test before using the form shown in 
(138) is 


Н,:с,-с (134) 


This test may be developed in a manner similar to that used in the 


other examples. We minimize x’ subject to the condition that Hy is 
true, i.e. we minimize 


по»: ¥u—a—b(X,.—X..) *c(x,—X,))? (135) 


with regard to the parameters a 


; b апас, to obtain the relative mini- 
mum value of x?, x?, 


say. The estimates of a and b are, in this case, 


the same as those given in equations (129) and (130), respectively. 
The other values are as follows: 


| о (У ad 
> 2 УХ, == 5 
eae Б? т, 6 
с (EX (136) 
Z|xxi-- | 
$ t п; 
х= 242, say (137) 
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(ZY4GX | 
~ (ха Уа] = 


п, 


where ха =>) 


ы Я (Хи) 
a 


п; 


" (Хх. Уа) E 
> (Хит) —— 


Ns 


= (138) 
(X) 
2| Xxi--—— 
5 t п; 
To test the hypotheses Нә : с, =c, we calculate 
% 2 
==} емі а (139) 
ті по 


and refer to Fisher’s tables of z with degrees of freedom 71 =k—1 апа 
m —N-—k-—2. In the evaluation of x? and xj, we need the quantities 
shown in the following table (calculated from the data in Table XL). 


TABLE XLII 
QUANTITIES REQUIRED IN THE EVALUATION OF Ха? AND ХІ? 
Уха E 
Уү,)(ХХ NY i (СХи)(2 Ка) 
Class G gr g С i (EX, |XXaYa- + 2 
"ns "ns m t t ns 
ms 
1 105,538.9091 26,096.4848 1,933.5152 2,974.0909 
2 84,740.0000 20,628.5714 2,121.4286 3,183.0000 
3 123,586.0000 33,102.2258 2,184.7742 4,483.0000 
4 187,920.8824 52,418.3824 4,218.6176 7,604.1176 
5 81,218.3226 18,048.5161 .3,287.4839 5,466.6774 
Total 583,004.1141 150,294.1805 13,745.8195 23,710.8859 


Substituting thése values in equations (132) and (138), we find 
xi =37,912.5059 
xb- 446.0445 
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and for substitution in equation (139) we calculate 


xb _ 446.0445 111.5111 
nı 5 

ха _ 37912.5059 _ 955 7500 
по 150 


Since xi/n, is actually less than x2/z,, it is clear that the departures 
from the common relationship are not significant. We may accept the 
hypothesis И», therefore, and conclude that there is a common within 
classes relationship or regression. 

Since the assumption с, = с is valid, we may present the final results 
of our analysis in the form shown below; this is the form generally used 
for analyses of this type. 


TABLE XLIII 


ANALYSIS OF VARIANCE AND COVARIANCE OF SCORES or PUPILS 
ON THE Two MENTAL Tests 


" Sum of Squares|Sum of Squares Sum of 

Variance аг. Second Test First Test Products 
Between Classes . . 4 22,141.6675 4,519.3015 9,868.7256 
Within Classes. . . | 152 78,667.2115 13,745.8195 23,710.8859 
Tora 722% | 150 100,808.8790 18,265.1210 33,579.6115 


From the results we obtain the values shown in the following table. 


TABLE XLIV . 


REGRESSION COEFFICIENTS, CORRELATION COEFFICIENTS AND ADJUSTED 
Sum or Squares FOR SECOND TEST 


Regression | Correlation Adjusted Analysis for Second Test 
Factor Coefficient | Coefficient 


d.f. | Sum of Squares | Mean Square 


Between F 

Classes 2.184 0.987 3 591.4911 197.164 

Within м 

Classes ) 1.725 0.721 151 37,767.0593 250.113 
Total 1,838 0.783 154 38,358.5504 — > 


From the adjusted analysis we find, as is to be expected, that the 
differences between the classes on the second test are not significant 
when corrected for the differences found on the first. 
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From the values of the correlation coefficients, we see that the 
relationships for between and within classes are very different; this 
also appears, of course, when we consider the regression coefficients. 
The relationships have been shown graphically in Figure 1; the regres- 
sion lines shown are those determined by the regression coefficients 
given in Table XLV. 


(50 4 
ө General Mean for Grade 


9, Mean for s-th Class 


Regression within s-th 
== == Class 


140 Common within Classes 
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Score on First Test 


Ficure 1 
Relationship between Scores of Pupils on two Mental Tests 
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TABLE XLV 
REGRESSION COEFFICIENTS 


Factor Regression Coefficients 


Between Classes b —2.184 


£171.538 
са=1.500 
су=2.052 


Within Classes 
с4=1.803 


с5 = 1.663 


с =1.725 


Total с1=1.838 


It is clear that the three general relationships: (1) between means 
of classes, (2) within classes, and (3) total for all classes, are very dif- 
ferent. The choice of a particular one for use will be determined 
mainly by the nature of the problem considered; for instance, if we are 
interested in the relationship within classes we shall choose the second 
one, с=1.725, and ignore the others. The usefulness of an analysis 
of this kind lies in the fact that it shows clearly the different relation- 
ships, and, at the same time, enables us to choose the appropriate 
measure for use in the solution of our educational problem. 


Note: 

In some cases we may wish to determine whether our measure of 
the relationship, the regression coefficient, is significantly greater than 
zero. As this problem is related to those discussed above, we shall 
consider here the test of the corresponding hypothesis for a particular 
class, the first, say. The hypothesis we wish to test, therefore, may 


be written 
Но: 6120 (140) 
The test of Но is developed by the usual method. We denote by 
x -Ai»-4-aQr-x))? (141) 


(see equation (126)) and minimize x?: 
(1) with regard to d and c, to obtain the absolute minimum value 
of x”, ха; 
(2) with regard to the parameters remaining if Hy is true, i.e. if 
¢,=0, to obtain the relative minimum value of x?, x2. 
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We obtain : 


EX) у) | 
ы ба“ em = 
Xa = 2201 - 142 
t n (xy n 
xxi-- 
t п 
2 2 2 
Xr = Ха +Х0, 
== Xatxo, say (148) 
(Sy) 222) | 
Eb -— — 
2 L 
y= 144 
ox? ш 
zxi-J 
т n 
and denotes the number of pupils in the class considered. 
To test the hypothesis Ho, we calculate 
2 2 
z=} log, pa ха) (145) 
ny по 


and refer to Fisher's tables of z with degrees ‘of freedom 7-1 and 
т=п—2. For the first class we have 

х2 =7671.8638 

xà =4574.6817 

n=33 


4574.6817 
z=} log, {7671.8638} =1.459 
31 

e 
From Fisher's tables, entered with degrees of freedom пл =1 and m=31, 
we find that z is greater than the 1% point. We reject the hypothesis 
Но, and conclude that the regression coefficient is significantly greater 
than zero. 


The tests of significan 
ered above may be developed in a similar manner. 


ce of the othér regression coefficients consid- 


Example 11. Reliability of Mental Tests: Relationship between the 
Sampling Unit and the Estimates of Reliability of the Test. 
ing the reliability of a mental test, we meet with а diffi- 


In determin 
the previous example. The tests 


culty similar to that considered in 
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are administered to groups of pupils, the school class is generally the 
unit chosen, and these groups form our sampling units. Estimates of 
the reliability of the test may be calculated separately for each class, 
or other such unit, and these results may be interpreted without diffi- 
culty as the statistical and sampling units are the same. We may, on 
the other hand, wish to use the same data in estimating the reliability 
of the test for a larger unit, say the grade. Since it will be necessary 
to combine the original data into larger groups, the statistical and 
sampling units will no longer be the same and the interpretation of the 
results may be difficult. In this example we shall consider the prob- 
lems of determining: (1) the effect of combining the data on our esti- 
mates of reliability, (2) the conditions which must be satisfied before 
the results may be combined, and (3) the interpretation of the results. 
(The data used in illustrating the methods refer to the scores of high 
school pupils on two forms of a mental test; the coefficients given 
below, therefore, are the Comparable Forms estimates of the relia- 
bility of the test. 

The problem of determining which group—the class, grade or 
school as a whole—should be chosen as the unit is related to those 
stated above. It is, however, an educational rather than a statistical 
problem and will not be considered here. A more detailed study oi 
this particular problem, using methods similar to those developed here, 
will be published later. 

As the arguments are easier to follow w 
case in mind, we shall begin with the analysis of the results for a group 
of Grade XI pupils. The original data, in summary form, for the four 
classes in this grade are given in the following table. 


hen we have a particular 


TABLE XLVI 
SconEs or GRADE XI Рорпв on Two FORMS or A MENTAL TEST 
Number Sum of Scores Sum of Squares Sum 
Class of of Scores of 
(5) Pupils Products 
Ns Ist Form | 2nd Form | 1st Form | 2nd Form 

1 32 1,180 1,352 48,092 | 60,402 53,265 

2 21 632 782 20,436 31,030 24,984 

3 41 1,084 1,292 32,062 44,064 36,969 

4 21 495 671 13,465 24,161 17,634 
Total 115 3,391 4,097 114,055 159,657 132,852 


Let us consider first an analysis of variance and covariance of these 
scores similar to that considered in the previous example. Applying 
the L; criterion discussed in the other examples, we find that the classes 
are of equal variability; we may, therefore, present the results in the 
form shown below. 


TABLE XLVII 


ANALYSIS OF VARIANCE AND COVARIANCE OF SCORES OF 
Рорп ON THE Two Forms 


" Sum of Squares em ef Correlation 
Гаги df. t ient 
алысы First Form |Second Form iod T But 

Between Classes . 3 2,870.1804 | 2,435.9160 | 2,557.0634 0.967 


Within Classes . | 111 | 11,194.5500 | 11,261.0058 | 9,486.8757 0.845 


Total . . . | 114 | 14,064.7304 | 13,696.9218 | 12,043.9391 0.868 


The correlation coefficients shown in the last column of the table are 
the measures usually employed in estimating the reliability of the test. 
The difference between the reliability coefficients for “within classes", 
r —0.845, and the “total”, 7 =0.868, is a measure of the effect of com- 
bining the data into а larger group. The latter coefficient refers to the 
reliability of the test for the grade, and is larger because of the inclusion 
of the very significant differences "between classes. 

This analysis, therefore, gives us a measure of the effect of com- 
bining the results. We can see more clearly what is happening, how- 
ever, if we consider analyses of the type shown in Example 2. The 
results for each class, presented in a form similar to that shown in 
Tables V and VII, are given m Table XLVIII. Underlying these 


analyses is the assumption that we may write 

Хы=А +В, Dy Cat 2ui 
where Хы denotes the score of the t-th individual in the s-th class on 
the ¿th form of the test; і-1, 25-4, 2, «a рісі, 2, oles afi 
k denotes:the number of classes and л, the number of pupils in the s-th 
class. A is a measure of the common ability of all individuals in the 
grade and is defined as the mean of the scores for all individuals, forms 
and classes; B, is a measure of the ability of the s-th class; D,; is a 
measure of the difference between the scores on the two forms of the 
test, ie. the practice effect, for the s-th class; Са is a measure of the 
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` results. 


ability of the ¢-th individual in the s-th class; 2,; represents the errors 
of measurement. It follows from the definition of A that 


SB, =0 
XD,-0 (147) 
> Са =0 

t 


In combining the results, we also assume that 


(1) z, is normally distributed about zero with constant stan- 
dard deviation, с say, constant for all classes ; 


(2) Cu n normally distributed about zero with constant stan- 
dard deviation, c, say, constant for all classes. 


If these assumptions are not satisfied, then, instead of с and c,, we 
shall have c, and с, respectively, i.e. standard deviations constant 


for each class only. We must, therefore, test the hypotheses 


Р H, :с;=с (148) 
апа 
Р авс: (149) 
6, =o 


(see equations (64) and (68) of Example 4) before combining the 


The tests of these hypotheses are the same as those discussed in 
Example 4 where the problem was similar. The relevant data and 
the method to be followed in evaluating log Гл, where Ly 1 the criterion 
used in testing these hypotheses, are shown in Tables XLIX and L. 
For the test of the hypothesis Hi, we have Ly =0.974 and from Nayer's 
tables, entered with k =4 and mean degrees of freedom f =27.8, we find 
that L, is greater than the 5% point. ~ We may accept the hypothesis 


. =p, 
Ні, and proceed to the test of the hypothesis На: G =. For the 
s 


test of this hypothesis, we have L,=0.983 and referring to Nayer's 
tables with &—4 and mean degrees of freedom f=55.5 we find that Lı 
is greater than the 5% point. We may, therefore, accept both Н and 
Н, and, since our assumptions are satisfied, we may combine the 


results. 
87 


TABLE XLIX 
TEST or THE НУРОУНЕЗ5 Hi:¢;=0; EVALUATION OF LOG L, 


Ns log ms ns log ns | 6% log 6% ns log 0% 


64 1.80618 =з 519.7500 2.71579 — 
213.2857 2.32896 — 


82 1.91381 — | 566.3902 2.75312 E 
| 


42 1.62325 E 


42 1.62325 == 441.4762 2.64491 јен 

230 Ens log л; =408.88094 | 1,740.9021 Ens log 6’; =608.46894 
5 5 
log 11 =1.98874 
TABLE 1, 
TEST or THE НуРОТНЕ515 Их: (еш ; EVALUATION OF Loc L; 

ns log л; ns log ns | 6, | log 0; ns log Os 

64 1.80618 —— | 7,859.5000 3.89539 — 

42 1.62325 -- | 3,325.6190 3.52187 === 

82 | 1.91381 | -- | 6,752.3415 3.82945 = 

42 | 1.62325 | — | 4518.0952 | 3.05405 -- 
230 | | | 


Ens log п; =408.88094 | 22,455.5557 | Хп; log 0;=864.74030 


Although it is not necess 
to consider the test of an 
results. It w 
cussion, we h 


ary to do so at this stage, it is convenient 
other hypothesis before we combine the 
ill be seen that in equation (146), and subsequent dis- 
ave considered the practice effect, Юу, separately for 
each class. It is possible that the Practice effect is constant for all 
classes, i.e. that 2,; =D;, say, forall s. We may develop а test of this 
hypothesis 

Hy: Da=D (150) 
by the usual method. Denote by 


x -EXX|x,, —4-B,-D,—C,]? (151) 
and minimize x': (1) with regard to all the parameters in (151) to 
obtain the absoluté minimum value of х, x2; and (2) with regard to 


ВА 


the parameters remaining if H is true, to obtain the relative minimum 
value of x°, х2. 


We obtain 
' ‚ EG Xr 
xi2X| iXX- Xa) - — (182) 
> t 2n, 
(see equation (46) of Example 2) 
х= хіх, say (153) 
where 
DEXa Bw 
х => £ l a 5- 
s п, 2n, 
x(Zxxa]* (УБ Хај“ 
lum Seeks (154) 


Уп, 2d, 


To test the hypothesis Из, we calculate 


2= log. [e m (155) 

ni то) 
^and refer to Fisher's tables of z with degrees of freedom n, =k —1 and 
по У (п,—1). The results of the analysis may be shown as in the 


following table: 


Test oF Hy: Ба = Рі; ANALYSIS OF VARIANCE OF СомвіхЕр RESULTS 


| 
+. ciance uf. | Sumef Mean 
Variances о ic | Squares Square 
Due to Common Practice Effect . 1 | 2,167.1130 | 2,167.1130 
е 3 95.9848 31.9949 


Due to Departures from Common 
Practice Effect 


Between Means of Classes 3 5,210.1117 | 1,736.7039 


11 20,714.6536 186.6185 
111 1,740.9021 15.6838 


Between Individuals (Within Classes) 


Error (Within Classes) 


| 
њ | 
| 
|. 
| 
| 
| 
| 


Тога! 229 29,928.7652 


We have 
95.9848 


740.9021 


Substituting these values in (155), we obtain 


31.9949 
=1] ——— >? =0.356 
р Hog. (598) 


and from Fisher’s tables, entered with degrees of freedom 7, =8 and 
пз=111, we find that z is less than the 5% point. We accept the 
hypothesis Hs, therefore, and conclude that the practice effect is 
constant for all classes. Our final analysis will be as shown in Table 
ЕН, 


TABLE LII 
FINAL ANALYSIS OF VARIANCE OF COMBINED RESULTS 
Variance df. Sum of Mean 
Squares Square 
Due to Common Practice Effect. . . . 1 2,167.1130 | 2,167.1130 
Between Means of Classes . . . . š 3 5,210.1117 | 1,736.7039 
Between Individuals (Within Clases) . . 111 20,714.6536 186.6185 
Error (Within Classes) . . |, a ж 114 1,836.8869 16.1130 
Ty en . | 7229 29,928.7652 == 


If we had ignored the differences between the classes and analyzed 


the results for the grade ds a whole, we would have found the results 
given in the following table: 


TABLE LIII 


ANALYSIS OF VARIANCE OF SCORE ДЕ PUPILS; FOR THE 
GRADE AS A Unit 


Variance р df. Sum of Mean 
Squares Square 


Between Forms , 1 2,167.1130 | 2,167.1130 


114 25,924.7653 227.4102 


Error (Within Classes) . . , DE m. 114 1,836.8869 16.1130 


Between Individuals 


Total 229 29,928.7652 = 


differences between individuals within classes. In conditions similar 
to those existing here, the effect of using the larger unit is exactly this 
merging of the two factors. The analysis shown in Table LII, there- 
fore, gives us a measure of the effect of combining the results into a 
larger unit. 

If the practice effect is not constant, but the other conditions are 
satisfied, then the analysis shown in Table LI will give us the desired 
measure of the effect of combining the results. In this case, of course, 
both the error and the between individuals variance in the combined 
analysis of the scores of all pupils in the grade, as in Table LIII, will 
be increased. 

Let us consider, finally, the interpretation of the results for a case 
where the conditions are not satisfied. The data given below in sum- 
mary form refer to the scores of pupils on the two forms of the mental 
test considered above, but the unit in this case is the grade, not the 


class. 
TABLE LIV 


Scores or Pupits ом Two Forms or A MENTAL Test (Bv GRADES) 


Sum of Squares 


Number Sum of Scores 
= Pupils т сыры Pes 
se | Fe | See | ecm eae 

x 157 4,062 4,784 119,544 | 164,040 137,593 

XI 115 3,391 4,097 114,055 159,657 132,852 
XII 115 4,484 5,025 194,382 237,933 213,023 

^ XIII 41 1,858 ^ 2,095 91,910 113,661 101,311 
Total 428 13,795 16,001 519,891 675,291 584,779 

EJ 


The analyses by grades are given in Table LV; in these analyses, of 
course, we ignore the effect of combining the results for all classes 
withina grade. Asin the previous case, we test the hypotheses H, and 
Н, before considering the final combination of the results for the school 
as a whole. From the values given in Tables LVI and LVII, we 


find that: 
(1) For the test of the hypothesis Hi : о, =0, we obtain Гл =0.999 
and from Nayer's tables, entered with k=4 and mean degrees of 
freedom f =106, we find that Гл is greater than the 5% point. 
We may accept the hypothesis Н, and, as far as the error term 
is concerned, combine the results. 
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TABLE LVI 
Test or THE HvrorHESIS Hi: 0; —60; EVALUATION OF LOG L, 


Hs log т; ns log ns 6; log 6% ns log 6's 
314 2.49693 — 2,538.8598 3.40464 — 
230 2.36173 — 1,836.8869 3.26408 A 
230 2.36173 === 1,861.9738 3.26997 -- 

82 1.91381 = 789.5122 2.89736 — 
856 zs log п; =2,027.36424 7,027.2327 Уи; log 9's =2,809.47198 

s 
log L,— 1.99936 
TABLE LVII 
Test оғ THE HYPOTHESIS Hz: несі EVALUATION OF 1.06 Ја 
оз; =o 

ns log ns as log ns 9; log 0s ns logüs 
314 2.49693 — 32,714.5732 4.51474 — 
230 2.36173 — 27,761.6522 4.44345 — 
230 2.36173 —— 37,907.3391 4.57872 --- 

82 1.91381 --- 14,322.4878 4.15602 —— 
856 Zn, log п; =2,027.36424 112,706.0523 | Zu; log = 3,833.52110 

5 5 
log Та =1.99051 
(2) For the test of the hypothesis H2: iS ы, ‚ we obtain Lı =0.978 
‚ = 


entered with k=4 and mean degrees 
d that Lı is less than the 1% point. 
therefore, and conclude that the 


and from Nayer's tables, 
of freedom f =212, we fin 
We reject the hypothesis 2, 


standard deviation, dcs, varies from 
bined analysis, for the test of the hypothesis На: Ds; =D;, 


will be in the form shown in "Table LVIII. 
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grade to grade. Our com- ' 


| 


TABLE ТУПЛ 


TEST or THE HyPorHESIS Ha: Dsi — Di; ANALYSIS OF VARIANCE 
OF COMBINED RESULTS 


Variance d.f. Sum of Mean 
Squares Square 
Due to Common Practice Effect 1 5,685.0887 5,685.0887 
Due to Departures from 
Common Practice Effect 3 99.6784 33.2261 
Between Means of Grades 3 39,539.7599 13,179.9200 
Grade X 156 30,175.7134 193.4341 
Between 
Grade XI 114 25,924.7653 227.4102 
Individuals 
Ч Grade XII 114 36,045.3653 316.1874 
in 
Grade XIII 40 13,532.9756 338.3244 
Error (Within Grades) 424 7,027.2327 16.5737 
Total 855 158,030.5793 


For the test of the hypothesis На: Ра 2 Dj, we have 
x; =7027.2328 


xi= 99.6784 
n; =3; по = 494 
and we find 
33.2261 
=} log, 4 — $ =0,34 
ал (Bat um 


From Fisher's tables, entered with degrees of freedom m,=8 and 
по = 424, we find that z is less than the 5% point. We accept the hypo- 
thesis H5: D,;=D;, and conclude thatthe departures from the com- 
mon practice effect are not significant. We may, therefore, present 
our final analysis in the form shown in Table LIX. 
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TABLE LIX 


FINAL ANALYSIS OF VARIANCE OF THE COMBINED RESULTS 
(ALL GRADES) 


Variance d.f. Sum of Mean 

Squares Square 
Due to Common Practice Effect 1 5,685.0887 5,685.0887 
Between Means of Grades 3 39,539.7599 13,179.9200 
Bieten Grade X 156 30,175.7134 193.4341 
Dalda Grade XI 114 25,924.7653 227.4102 
D Grade XII 114 36,045.3658 316.1874 
Grade XIII 40 13,532.9756 338.3244 
Error (Within Grades) 427 7,126.9111 16.6907 

"Total 855 158,030.5793 


The effect of ignoring the differences between grades and analyzing 
the results for the school as a whole is shown in Table LX. 


TABLE LX 
> EFFECT or COMBINING THE RESULTS FOR ALL GRADES ON THE 
ANALYSIS OF VARIANCE 
Variance d.f. Sum of Mean 
Squares Square 
Between Means of Grades 3 39,539.7599 13,179.9200 
Grade X 156 30,175.7134 193.4341 
Between 
Grade ХІ 114 25,924.7653 227.4102 
Individuals 
А Grade XII 114 36,045.3653 ‚ 816.1874 
m 
Grade ХІІІ” 40 13,532.9756 338.3244 
B Individual 
Total {б ай Grades a 427 | 145,218.5795 340.0904 


The differences between grades are shown in the first five rows; if we 
combine these results we assume tha. there is a common variance, 
which clearly is not the case, and use the mean square in the last row, 


340.0904, as the estimate of it. 
The interpretation of the resu 
We have a significant practice effect, 
the ability of pupils in different gra 
between the variability of the scores o 
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Its given in Table LIX is not difficult. 
significant differences between 
des, and significant differences 
f pupils in different grades. If 


we assume that the discriminating power of a mental test may be 
measured by the degree to which it differentiates between the indi- 
viduals of a particular group, it is clear that the test considered is more 
suitable for use in the higher grades. Finally, we find that the errors 
of measurement by means of the test are constant for all grades. The 
probable error of the score of any individual in the groups considered 
дол О шант RTI ТЕ ВО 
= 2.8 score units. 
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APPENDIX A 


TABLE VI 


5 Per Cent. POINTS оғ тив DISTRIBUTION OF 2 


Values of nı 


1. 2. % 4 dy 


| | 5. | 6 8. 12. 21 © 
2-6479 | 2-6870 | 2-7071 | 2-7194 | 2.7270 | 2-7380 | 2-7484 | 2-7588 | 2-7693 
1-4722 | 1.4705 | 1-4787 | 1-4800 | 1-4808 | 1-4819 | 1-4830 | 1-4840 | 1185 
1-1954 | 1-1137 | 1-1051 | 1-0994 | 1-0953 | 1-0899 | 1-0842 | 1-0781 | 1-0716 
-0690 | -9429| -9272| -9168| .9093| -8993 | -8885 +8639 
-8777 | -8441| -8236| -8097| -7997| -7862| -7714 +7368 
:8188 | .7798| -7558| -7394| .7274| -7112| -6931 :6100 
7777 | -7347| -7080| -6896| -6761| -6576| -6369 +5862 
“7475 -6725 | -6525| -6378| -6175| -5945 +5371 
7242 | +674 -6450 | -6238| -6080| -5862| -5613 -4979 
-7058 | -6553| -6232| -6009| -5843| -5611| -5346 -4657 


11 | -7889| -6909| -6387 
12 | -7788| -6786| -6250 
#13 | -7703| -6682| -6134 
1 -7630 | 6594 | -6036 
7568 | -6518| -5950 
16 | -7514| бил 
17 | -7466 | -6393 
18 | -7424| -6341 
19 | 7386 | 6295 
20 | -7352| .6254 | -5651 


-5126 | -4795| 4387 
-4941 | -459 156 
4785| 419. | +3957 
-4649 | -4269| -3782 
-4532 | -4138| -3628 
-28| 4022] -3490 
-4337 | -3919| -3306 
4255 | :3827 | + 

-4182| -3743 
4116 | -3668 


mE 


Values of 2; 


| 
“5 1 
21 | -7322| .6216| -5612| 25219 .4055 | -3599| 2971 
22°} .7294 | -6182 | -5574| -5179 -4001| -3536| -2892 
23 | -7269| -6151 5140 | .3950 | -3478| -2813 
| 24 | .7246 | -6123| - 5106 № -4817 4953 | -3904 | -3125 | -274 
25 | -7225| .6097 | -5478| -5074| +4783 4244 | -3862| .3376 | 268: 
әб | -7205 | -6073| -5451| -5045| 4752 -4209| -3823| -3330| 22 
2. 


97! .7187| -0051| -5427| -5017! > 1723 4176 | -3786 | +3287 
ов | -7171 | -6030| -5403| -4992| -4696 A471] 4146 | 3752 | 3248 
| од | -7155| -6011| -5382| -4969| +4671 | - 134] -4117| -3720| -3211 
30 | -7141| -5991| -5362| +4947 | -4648 1120} -4090| 3691 | 3170 


во | -6933| -5738| -5073| -4632| -4311 4064 | -3702| +3255] 2654| -1644 


-4319| -3974 | -3706 | -3309 :2804 | -2085| 0 


со -6729 | -5486 |. -4787 | 


from "‘Statistical Methods for Research Workers” 
by Oliver & Boyd, Edinburgh, and attention is 


This table is reproduced by permis ion 
"ables" by Professor R. А. Fisher and Г. Yates, 


by Professor R. A. Fisher, published а 1 
drawn to the larger collection in "54 tical 
published at 12/6 by Oliver & Boyd, Edinburgh. 
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TABLE VI.—Continued 


1 PER CENT. POINTS OF THE DISTRIBUTION OF 2 


554 Values of zi. 

i 2. EN 4. 5. 6. 8. 12 24. ©. 
1 |4-1535 | 4-2585 | 4-2974 | 4-3175 | 4-3297 | 4.3379 | 4-3482 | 4-3585 | 4.3689 | 4.3704 
2 | 2-2950 | 2.2976 | 2.2084 | 2.2988 | 2.2901 | 2.2992 | 2.2994 | 2.2997 | 2.2999 | 2-3001 
3 | 1-7649 | 1-7140 | 1-6915 | 1-6786 | 1.6703 | 1-6645 | 1-6569 | 1.6489 | 1-6404 | 1-6314 
4 | 1-5270 | 1-4452 | 1-4075 | 1-3856 | 1-3711 | 1-3609 | 1-3473 | 1.3327 | 1.3170 | 1-3000 
5 | 1-3943 | 1.2929 | 1.2449 | 1.2164 | 1-1974 | 1.1838 | 1-1644 | 1.1457 | 1-1239 | 1.0997 
6 | 1-3103 | 1-1955 | 1.1401 | 1-1068 | 1-0843 | 1-0680 | 1.0460 | 1-0218 | -9948 | -9643 
7 | 1-2526 | 1.1281 | 1-0672 | 1.0300 | 1-0048 | -9864| -9614| -9335| .9020| -8658 
8 | 1-2106 | 1-0787 | 1.0135 | -9734 | -9459| .9259| -8983| -8673| .8319| -7904 
9 | 1-1786 | 1-0411 | -9724 | -9299| -9006| -8791| -8494| -8157| -7769| -7305 
10 | 1-1535 | 1.0114 | -9399| -8954| -8646| -8419| -8104| .7744| -7324| -6816 
11 | 1-1333 | -9874| -9136| -8674| -8354| -8116| -7785| -7405| -6958| -6408 
12 | 1.1166 | 9677 | -8919| .8443 | -8111| -7864| -7520| -7122| -6649| -6061 
13 | 1-1027 | +951 | -8737| -8248| -7907| -7652| -7295| -6882| -6386| -5761 
14 | 1-0909 | -9370 | -8581 | -8082| -7732| -7471| -7103| -6675| -6159| -5500 
# | 15 | 1:0807 | :9249| .8448| -7939| -7582 | -7314| -6937| -6496| -5961| -5269 
`5 | 16 | 10719 | :9144| .8331 | -7814| -7450| -7177| -6791| -6339| -5786| 35064 
g| 17 | 1-0641 | -0051 | .8229| -7705| -7335| -7057| -6663| -6199| -5630| -4879 
= | 18 | 1-0572| -8970| -8138| -7607| -7232 | -6950| -6549 :6075 | -5516| -4712 
> | 19 | 1-0511 | -8897| -8057 | -7521| -7140 | -6854| .6447| -5964| -5366| -4560 
20 | 1-0457 | -8831| -7985 | -7443| -7058| .6768| -6355| -5864| -5253| -4421 
21 | 1-0408 | .8772| .7920 | -7372| -6984| -6690| .6272| -5773| -5150| -4294 
22 | 1-0363 | -8719| -7860| -7309| -6916| -6620| -6196| -5691| -5056| -4176 
23 | 1-0322 | .8670 | -7806 | -7251 | -6855| %555| -6127| -5615 -4969 | -4068 
24 | 1.0285 | .8026 | -7757| -7197| -6799| 6496| -6064| -5545| .4890| -3967 
25 | 1:0251 | -8585| -7712| -7148| -6747| -6442| -6006| -5481| .4816 | -3872 
26 | 1:0220 | +8548 | -7670| -7103| -6698 .6392| -5952| -5422| .4748| -3784 
27 | 1.0191 | .8513 | .7631 | -7062| -6655| .6346 | -5902 :5367 | -4685| +3701 
28 | 1:0164 | -8481 | -7595| .7023 | -6614| -6303 -5856 | -5316 | -4626| -3624 
29 | 1.0139 | „8451 | -7562 | -6987| -6576| -6263| -5813| .5269 | -4570| -3550 
30 | 1.0116 | -8423| -7531| .6954 | .6540| -6226| -5773| -5224| .4519| -3481 
60 | -9784| .8025| -7086 | -6472| -6028| -5687| -5189 4574 | -3746 | +2352 

со | 9462| -7636| -6651| -5999| -5522| .5152| -4604| -3908| -2913| 0 


This table is reproduced by permission from “Stat istical 
by Professor R. A. Fisher, published at 15/- by Oliver & Bo: 
drawn to the larger collection in “Statist 
published at 12/6 by Oliver & Boyd, Ed 
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TABLE 103—525 (ROMAN TYPE) AND 1% (BOLD FACE TYPE) POINTS FOR THE DISTRIBUTIO? 
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TABLE 10.3—5% (ROMAN TYPE) AND 1% (BOLD FACE TYPE) POINTS FOR THE DISTRIBUTION OF F 
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TABLE 103--5% (ROMAN TYPE) AND 1% (BOLD FACE TYPE) POINTS FOR THE DISTRIBUTION OF 
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APPENDIX C 


TABLE IV. 5 per cent. limits for Li. 
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Taste V. г per cent. limits for Іл. 


6 7 8 9 11 | 14 | 19 | 29 | 59 | œ 


-141 | -284 | -398 | -485 | -551 | :603 | -645 | -678 | -730 | -783 :836 | -890 | -945 | 1: 

-162 | -314 | -429 | -514 | -578 | -628 | -667 | -699 | -748 | -798 -848 | -898 | -949 | 1-000 
-188 | -345 | -459 | -542 | -604 | -652 | -689 | -719 | -705 .812 | -859 | -906 | -953 | 1-000 
:210 | -370 | -484 | -565 | -624 | -670, -706 | -735 | -779 -823 | -867 | -911 | -956 | 1-000 
.230 | -391 | -504 | -583 | -641 | -685 | -720 | -748 | -789 -832 | -874 | -916 | -958 | 1-000 
-246 | -409 | -520 | -597 | -654 | -697 | -730 | -757 | -798 -839 | -879 | -920 | -960 | 1-000 
-260 | -424 | -534 | -610 | -665 | -707 | -740 | -766 -805 | -844 | -884 | -923 | -962 | 1-000 
-273 | -437 | -545 | -620 | -674 | .715 | '747 | 773 -811 | -849 | -887 | -925 | -963 | 1-000 
10 | -284 | -448 | -555 | -629 | -682 | -722 E 


~ 
ee | | 
со 
~ 
сл 
© 
EI 
ю 
© 
- 
© 
- 
to 
Em 
= 
to 
= 
о 
| $ 
© 
© 
= 
© 
= 8 
© 


12 | -303 | -467 | -572 | -644 | .696 | -734 | - џ к E б 
14 | 318 | 481 | -585 | .655 | -706 | -744 | -773 | -¥96 | -831 | -865 | -900 | -988 :967 | 1- 
16 | -331 | -493 | -596 | -665 | -714 | -751 | -779 -802 | -836 | -870 | -903 | -936 | -968 1-000 
18 | -342 | -504 | -605 | -672 | -721 | -756 | -784 -807 | -840 | -873 | -905 | -937 -969 | 1-000 
20 | -352 | .512 | -613 | -679 | 727 | -761 | -788 | -811 -844 | -876 | -908 | -939 | -970 1.000 . 
22 | -360 | -520 | -619 | -684 | -732 | -765 .792 | -814 | -847 | -878 -909 | -940 | -970 | 1-000 
24 | -367 | -526 | -624 | -688 | -736 |6768 -195 | -817 | -850 | 880 | -911 91 + 1:000 
26 | -373 | -532 | -629 | -693 | -740 [7:772 | -798 | -820 | -852 -882 | -912 | -942 | -971 | 1-000 
28 | -379 | -537 | -634 | -697 | -744 | -776 -802 | -823 | -854 | -884 :914 | -943 | -972 | 1- 
30 | 386 | 513 | 639 | -703 | -748 | -781 | -806 | -827 | -856 | -886 | -915 944 | - 1-000 
These tables are reprodyced by permission from "Statistical Research Memoirs", Volume 1, 
edited by J. Neyman aud E. S. Peoi and issued by the Department of Statistics, University 
of London, University College, London, England. 
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